UPDATING revision 1.372
1 $NetBSD: UPDATING,v 1.372 2024/11/22 16:07:32 riastradh Exp $ 2 3 This file (UPDATING) is intended to be a brief reference to recent 4 changes that might cause problems in the build process, and a guide for 5 what to do if something doesn't work. 6 7 For a more detailed description of the recommended way to build NetBSD 8 using build.sh, see the BUILDING file. 9 10 Note that much of the advice in this UPDATING file was written before 11 build.sh existed. Nevertheless, the advice here may be useful for 12 working around specific problems with build.sh. 13 14 Sections are marked with "^^^^^". After the section on "Recent changes" 15 are several sections containing more general information. 16 17 See also: BUILDING, build.sh, Makefile. 18 19 Recent changes: 20 ^^^^^^^^^^^^^^^ 21 22 20241122: 23 Changes to various build options require cleaning their objdir 24 in update builds: 25 26 cd external/bsd/file/lib && $TOOLDIR/bin/nbmake-$ARCH clean 27 cd external/bsd/libarchive/lib && $TOOLDIR/bin/nbmake-$ARCH clean 28 cd external/public-domain/xz/lib && $TOOLDIR/bin/nbmake-$ARCH clean 29 30 Changes to bsd.lib.mk to improve detecting differences in 31 shared library symbol exports may cause some .actsym files in 32 the objdir to become stale in update builds. 33 34 To work around this, delete all files matching `*.actsym' in 35 the objdir. For example: 36 37 find OBJDIR -name '*.actsym' -type f -print0 | xargs -0 rm 38 39 20241104: 40 Changes around the integration of the zstd compression 41 library may require manual deletion of some object directories 42 if you update from a build from a few days ago: 43 rm -rf DESTDIR/usr/lib/*/libarchive* 44 rm -rf DESTDIR/usr/lib/*/libmagic* 45 rm -rf DESTDIR/usr/lib/*/libzstd* 46 rm -rf DESTDIR/usr/lib/libarchive* 47 rm -rf DESTDIR/usr/lib/libmagic* 48 rm -rf DESTDIR/usr/lib/libzstd* 49 rm -rf OBJDIR/compat/*/*/external/bsd/file/ 50 rm -rf OBJDIR/compat/*/*/external/bsd/libarchive/ 51 rm -rf OBJDIR/compat/*/*/external/bsd/zstd/ 52 rm -rf OBJDIR/external/bsd/file/ 53 rm -rf OBJDIR/external/bsd/libarchive/ 54 rm -rf OBJDIR/external/bsd/zstd/ 55 rm -rf OBJDIR/external/gpl3/gcc/ 56 rm -rf OBJDIR/tools/gcc/ 57 58 20240926: 59 Building -current when running on amd64 from about 20240924 60 to about 20240926 will not work due to a "malloc_np.h" 61 error. Please downgrade to a version from 20240924 or older, 62 or upgrade to a newer version e.g. using a binary release from 63 the NetBSD build servers. 64 65 20240923: 66 Update builds will require cleaning the object directories for 67 jemalloc (like: "rm -rf $OBJ/external/bsd/jemalloc*") and the 68 libc dependency files (like: "rm $OBJ/lib/libc/jemalloc.*"). 69 On architectures with COMPAT libs, those also have to be cleaned, 70 e.g. on sparc64: "rm -rf $OBJ/compat/sparc64/sparc/external/bsd/jemalloc*" 71 and "rm $OBJ/compat/sparc64/sparc/lib/libc/jemalloc.*". 72 73 20240817: 74 Update builds will require cleaning in src/external/historical/nawk/bin 75 in order to remove the awk binary since now there is a subdirectory 76 called awk. 77 78 20240812: 79 Update builds will require removing the tools objdir for gdb 80 if building with MKCROSSGDB=yes. All architectures use gdb.old 81 now. 82 83 20240630: 84 Update builds will require removing the tools objdir for binutils 85 (as most architectures should use binutils.old now). 86 87 20240610: 88 Update builds are likely to require removing the objdir for 89 distrib/<arch>/cdroms - or at least references to libarchive 90 from within there after the recent libarchive update. 91 92 20240504: 93 Update builds on x86 require making the `cleandir' target in 94 lib/libm, or deleting the objdir for lib/libm. 95 96 20240423: 97 Update builds with x11 may fail after switching to Mesa 21. 98 99 20240410: 100 Update builds may require manually deleting 101 $DESTDIR/var/run/named (and, potentially, 102 $DESTDIR/var/run/lwresd) in order to avoid checkflist failure. 103 104 20240409: 105 32-bit compatibility libraries were moved into the base32 106 and debug32 sets. HTML man pages were moved into the manhtml set. 107 108 20230828: 109 If: 110 - you updated to current and ran postinstall between 20230826 111 and 20230828 inclusive, and 112 - you are not using anything outside the base system to 113 populate /etc/openssl/certs (e.g., manually adding 114 certificates or using ca-certificates or mozilla-rootcerts or 115 mozilla-rootcerts-openssl from pkgsrc), 116 delete /etc/openssl/certs and rerun postinstall or just 117 `certctl rehash'. 118 119 Otherwise, certctl(8) will think someone else has left 120 /etc/openssl/certs in the way and will avoid clobbering it on 121 upgrade from older NetBSD. 122 123 20230718: 124 build.sh will use mk.conf in the same directory as build.sh instead 125 of /etc/mk.conf, unless MAKECONF is already defined in the environment. 126 127 20230714: 128 Import of gcc 10.5 requires a clean build of at least tools/gcc 129 and external/gpl3/gcc* object directories. An update build 130 without cleaning will result in obscure failures in rare corner 131 cases. 132 133 20230604: 134 Building native kernels with the installed make(1) instead of 135 the $TOOLDIR/bin/nbmake-${arch} wrapper will fail if userland 136 has not been updated. Note that this is an unsupported build 137 configuration (but usually works if userland is -current enough). 138 139 20230523: 140 ctfmerge was fixed as macOS host tool. This may require a clean tools 141 rebuild to detect the update to tools/compat/configure. 142 143 20230515: 144 New openssl build requires removal of the crypto/external/bsd/openssl 145 and tests/crypto/libcrypto subdirectories in the ${OBJDIR} hierarchy. 146 Otherwise test results may be wrong due to wrong "golden" output 147 files being installed. 148 149 20230322: 150 Fixes for moved /lib/libisns* mean that builds will complain about 151 ./usr/lib/libisns* being missing until src/lib/libisns is forced 152 to (re)install the files, including the symlinks. The simplest 153 way is to clean this subdir before building. 154 155 20230112: 156 New binutils require an updated ld.elf_so. If you are doing 157 (unsupported) in-place self builds (with the -E flag to build.sh), 158 make sure to have installed latest ld.elf_so before you rebuild 159 userland. 160 161 20221116: 162 The addition to NetBSD's version of UFS2 of support for extended 163 attributes broke backward compatibility with previous releases 164 of NetBSD, so UFS2 has been restored to being compatible with 165 previous NetBSD releases by disabling extended attributes. 166 (Note that ACLs are implemented as extended attributes, so 167 this changes disables ACLs as well.) 168 169 Support for UFS2 with extended attributes is now available in a new 170 UFS variant called UFS2ea. If you have created extended attributes 171 in an original UFS2 file system then "fsck -p" will now fail due to 172 the unexpected presence of extended attributes and "fsck -y" will 173 remove all extended attributes. If you wish to preserve extended 174 attributes rather than delete them, there is a utility to convert 175 a UFS2 file system to UFS2ea and leave extended attributes in place, 176 but this should be used with caution since it will preserve any 177 extended attributes that have been corrupted by the backward 178 incompatibility too. 179 180 If you wish to use a UFS2ea file system as your root file system, 181 then you will need to update your boot loader to a version that 182 supports UFS2ea. 183 184 For more information, see: 185 https://wiki.netbsd.org/features/UFS2ea 186 187 20221111: 188 The new libdrm import worsened the conflict issues for the 189 kdump/ktruss ioctl, and i915 now conflicts with base, and has 190 been turned off. This will cause update build issues like: 191 192 kdump-ioctl.c:12175:143: error: 'DRM_IOCTL_I915_DESTROY_HEAP' 193 undeclared here (not in a function); 194 did you mean 'DRM_IOCTL_MODE_DESTROY_DUMB'? 195 196 You'll need to clean usr.bin/ktruss, usr.bin/kdump, and rescue. 197 198 20220921: 199 Since the kernel version was bumped to 9.99.100, the bootloader 200 must be updated on x86 or EFI boot platforms in order to load 201 modules from boot.cfg(5) or interactively at the bootloader 202 prompt. 203 204 The kernel can still load modules at runtime, with modload(8) 205 or modules.conf(5), with no bootloader update. This will not 206 affect release branches because it only applies to patch 207 numbers >=100. 208 209 On x86 (i386 or amd64) with BIOS boot, this requires copying a 210 new /usr/mdec/boot to /boot. You can build this in the destdir 211 by running `$TOOLDIR/bin/nbmake-$ARCH dependall install' in 212 sys/arch/i386/stand. 213 214 On platforms with EFI boot (including x86 with EFI boot), this 215 requires copying a new /usr/mdec/boot*.efi to the EFI/BOOT/ 216 directory on the EFI system partition. The boot*.efi file is: 217 218 aarch64 bootaa64.efi 219 amd64 bootx64.efi 220 arm bootarm.efi (32-bit arm) 221 i386 bootx32.efi 222 223 You can build this in the destdir by running 224 `$TOOLDIR/bin/nbmake-$ARCH dependall install' in 225 sys/stand/efiboot/boot*. 226 227 20220821: 228 Support for building extsrc/ has been deprecated. 229 EXTSRCSRCDIR and MKEXTSRC have been deprecated. 230 231 20220714: 232 Updates to xorg-server and associated drivers may cause builds to 233 fail. Cleaning both DESTDIR and the xorg build tree may be needed. 234 235 20220628: 236 Changes in the build of src/games/robots require a clean build 237 of that program. 238 239 20211116: 240 Changes in the xorg build require clean obj dirs for external/mit/xorg 241 (or a full clean/non-update build). 242 243 20211112: 244 Device tree sources were updated for evbarm. Some device nodes 245 (in particular, ld(4) devices on the ROCKPro64) will be renumbered. 246 247 20211014: 248 MKPIE default for aarch64 has changed, a clean build is required. 249 250 20210917: 251 Changed MNT_ACLS to default to POSIX1e ACLs instead of NFSv4 252 ACLs, to be compatible with FreeBSD. If you are using NFSv4 ACLs 253 and have "acls" in /etc/fstab you'll need to change it to "nfs4acls". 254 255 20210711: 256 Updated GMP sources may require cleaning in tools/gmp and/or in 257 external/lgpl3/gmp/lib, particularly if GCC itself does not build. 258 259 20210417: 260 GCC 10 was enabled for several platforms. If builds fail in either 261 tools/gcc or external/gpl3/gcc, first try cleaning those objects and 262 removing the $DESTDIR/usr/include/g++ subdirectory. 263 264 20201016: 265 MIPS kernel modules have been disabled until they work. This will 266 turn up in extra files in the DESTDIR, which should be cleaned. 267 268 20200925: 269 GNU MPC and MPFR have been updated. At least MPFR needs cleaning 270 in both the tools and external dirs. 271 272 20200912: 273 GCC 9 has arrived for Arm and x86 platforms, and will be coming 274 for several more, as will binutils 2.34. Clean as required. 275 276 20200907: 277 GCC 9 is coming, and binutils has been updated for MIPS. This 278 probably requires cleaning the tools/binutils. 279 280 20200811: 281 GCC updates may require cleaning the objdir. This may occur 282 due to GCC 7.5 update, GCC 8.4 move to gcc.old, or the upcoming 283 GCC 9 upgrade. 284 285 20200614: 286 blacklist* has been renamed to blocklist*. postinstall(8) 287 should handle the migration. 288 289 20200601: 290 Due to a mistake in LIBISPRIVATE handling, .so libraries were 291 created in the build directories and need cleaning. 292 293 20200311: 294 GCC 8 ports will need cleaning in src/tools/gcc and 295 src/external/gpl3/gcc due to GCC 8.4 update. 296 297 20191118: 298 More architectures were switched to gcc8: 299 i386, ia64 powerpc64, sparc, sparc64, arm 300 The same comments as in 20191022 apply. 301 302 20191112: 303 The LLVM update requires a clean rebuild for all architectures using 304 LLVM during the tools build phase (i386, amd64, aarch64). 305 306 20191022: 307 Ports amd64 and aarch64 switched to GCC 8.3 by default. 308 In-place ("expert mode", build.sh -E) builds are not supported 309 when going from a GCC 7 userland to GCC 8. Do a regular 310 build to a different DESTDIR (or preferably: build.sh -U) at least 311 once and install sets, or download comp.{tar.xz,tgz} from the 312 daily builds and install that before doing the next in-place build. 313 314 20191001: 315 GCC 8.3 was imported. Builds of src/tools/gcc may fail if 316 old builds with GCC 7 output now uses GCC 8. Clean this 317 directory, and also clean src/external/gpl3/gcc. 318 319 20190903: 320 Files with names that coincide with existing files' names on 321 case-insensitive file systems were inadvertently committed, for 322 radeon GPU firmware. We cannot mark these as obsolete for 323 postinstall to fix, so if you updated src since 2019-08-26, and 324 ran build.sh distribution or ran build.sh release, you must 325 manually delete the following files in your DESTDIR (which is 326 usually $OBJDIR/destir.$ARCH), or from / if you have installed 327 them: 328 329 /libdata/firmware/radeon/bonaire_ce.bin 330 /libdata/firmware/radeon/bonaire_mc.bin 331 /libdata/firmware/radeon/bonaire_me.bin 332 /libdata/firmware/radeon/bonaire_mec.bin 333 /libdata/firmware/radeon/bonaire_pfp.bin 334 /libdata/firmware/radeon/bonaire_rlc.bin 335 /libdata/firmware/radeon/bonaire_sdma.bin 336 /libdata/firmware/radeon/bonaire_smc.bin 337 /libdata/firmware/radeon/bonaire_uvd.bin 338 /libdata/firmware/radeon/hainan_ce.bin 339 /libdata/firmware/radeon/hainan_mc.bin 340 /libdata/firmware/radeon/hainan_me.bin 341 /libdata/firmware/radeon/hainan_pfp.bin 342 /libdata/firmware/radeon/hainan_rlc.bin 343 /libdata/firmware/radeon/hainan_smc.bin 344 /libdata/firmware/radeon/hawaii_ce.bin 345 /libdata/firmware/radeon/hawaii_mc.bin 346 /libdata/firmware/radeon/hawaii_me.bin 347 /libdata/firmware/radeon/hawaii_mec.bin 348 /libdata/firmware/radeon/hawaii_pfp.bin 349 /libdata/firmware/radeon/hawaii_rlc.bin 350 /libdata/firmware/radeon/hawaii_sdma.bin 351 /libdata/firmware/radeon/hawaii_smc.bin 352 /libdata/firmware/radeon/kabini_ce.bin 353 /libdata/firmware/radeon/kabini_me.bin 354 /libdata/firmware/radeon/kabini_mec.bin 355 /libdata/firmware/radeon/kabini_pfp.bin 356 /libdata/firmware/radeon/kabini_rlc.bin 357 /libdata/firmware/radeon/kabini_sdma.bin 358 /libdata/firmware/radeon/kaveri_ce.bin 359 /libdata/firmware/radeon/kaveri_me.bin 360 /libdata/firmware/radeon/kaveri_mec.bin 361 /libdata/firmware/radeon/kaveri_mec2.bin 362 /libdata/firmware/radeon/kaveri_pfp.bin 363 /libdata/firmware/radeon/kaveri_rlc.bin 364 /libdata/firmware/radeon/kaveri_sdma.bin 365 /libdata/firmware/radeon/mullins_ce.bin 366 /libdata/firmware/radeon/mullins_me.bin 367 /libdata/firmware/radeon/mullins_mec.bin 368 /libdata/firmware/radeon/mullins_pfp.bin 369 /libdata/firmware/radeon/mullins_rlc.bin 370 /libdata/firmware/radeon/mullins_sdma.bin 371 /libdata/firmware/radeon/oland_ce.bin 372 /libdata/firmware/radeon/oland_mc.bin 373 /libdata/firmware/radeon/oland_me.bin 374 /libdata/firmware/radeon/oland_pfp.bin 375 /libdata/firmware/radeon/oland_rlc.bin 376 /libdata/firmware/radeon/oland_smc.bin 377 /libdata/firmware/radeon/pitcairn_ce.bin 378 /libdata/firmware/radeon/pitcairn_mc.bin 379 /libdata/firmware/radeon/pitcairn_me.bin 380 /libdata/firmware/radeon/pitcairn_pfp.bin 381 /libdata/firmware/radeon/pitcairn_rlc.bin 382 /libdata/firmware/radeon/pitcairn_smc.bin 383 /libdata/firmware/radeon/tahiti_ce.bin 384 /libdata/firmware/radeon/tahiti_mc.bin 385 /libdata/firmware/radeon/tahiti_me.bin 386 /libdata/firmware/radeon/tahiti_pfp.bin 387 /libdata/firmware/radeon/tahiti_rlc.bin 388 /libdata/firmware/radeon/tahiti_smc.bin 389 /libdata/firmware/radeon/verde_ce.bin 390 /libdata/firmware/radeon/verde_mc.bin 391 /libdata/firmware/radeon/verde_me.bin 392 /libdata/firmware/radeon/verde_pfp.bin 393 /libdata/firmware/radeon/verde_rlc.bin 394 /libdata/firmware/radeon/verde_smc.bin 395 396 We will re-import these radeon firmware images another way 397 later. 398 399 20190727: 400 The uefi bootloader has gained tftp support and needs a clean 401 build. If you do update builds, manually clean its object 402 directory by something like: 403 cd sys/arch/i386/stand/efiboot && make clean 404 405 20190723: 406 The jemalloc allocator in libc is now built without extended 407 debugging (for performance reasons). In update builds make sure 408 to rebuild it completely, by removing all affected object files, 409 including compat builds, something like: 410 cd /usr/obj && find . -type d -name jemalloc|xargs rm -rf 411 412 20190207: 413 GCC 7 switched for many ports. Update builds are likely to fail. 414 415 Hints for a more successful build: 416 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 417 Use build.sh, but do not use its "expert mode": 418 This will automatically build the tools in the 419 correct order, and it will keep the tools and the 420 new build products from interfering with the running 421 system. This will allow you to ignore most of the 422 other advice in this file. 423 Build a new kernel first: 424 This makes sure that any new system calls or features 425 expected by the new userland will be present. This 426 helps to avoid critical errors when upgrading. 427 Use object directories: 428 This helps to keep stale object 429 files from polluting the build if a Makefile "forgets" 430 about one. It also makes it easier to clean up after 431 a build. It's also necessary if you want to use the 432 same source tree for multiple machines. 433 To use object directories with build.sh: 434 a) invoke build.sh with the "-M" or "-O" options. 435 To use object directories without using build.sh: 436 a) cd /usr/src ; make cleandir 437 b) Add "OBJMACHINE=yes" to /etc/mk.conf 438 c) Add "MKOBJDIRS=yes" to /etc/mk.conf 439 d) cd /usr/src ; make build 440 Note that running "make obj" in a directory will create 441 in obj.$MACHINE directory. 442 Build to a DESTDIR: 443 This helps to keep old installed files (especially libraries) 444 from interfering with the new build. 445 To build to a DESTDIR with build.sh, use the "-D" option. 446 To build to a DESTDIR without using build.sh, set the DESTDIR 447 environment variable before running make build. It should be 448 set to the pathname of an initially empty directory. 449 Problems: if you do not use build.sh, you might need to 450 update critical utilities without using DESTDIR since 451 nothing is executed from what is installed in DESTDIR. 452 (See critical utils, below.) 453 Build often: 454 This keeps critical utilities current enough to not choke 455 on any other part of the source tree that depends on up to 456 date functionality. If you use build.sh, you should not have 457 this problem. 458 459 What to do if things don't work: 460 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 461 When things don't work there are usually a few things that commonly 462 should be done. 463 1) make includes 464 This should be done automatically by make build. 465 2) cd share/mk && make install 466 Again, automatically done by make build. 467 468 Failsafe rebuild of a small part of the tree: 469 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 470 To make sure you rebuild something correctly you want to do 471 something like the following: 472 1) Make sure the includes and .mk files are up to date. 473 2) Make sure any program used to build the particular 474 utility is up to date. (yacc, lex, etc...) 475 3) cd ...path/to/util... 476 make cleandir 477 rm ...all obj directories... 478 make cleandir # yes, again 479 make obj 480 make depend && make 481 482 Failsafe rebuild of the entire tree: 483 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 484 If you really want to make sure the source tree is clean and 485 ready for a build try the following. Note that sourcing /etc/mk.conf 486 (a make(1) Makefile) in this manner is not right, and will not work 487 for anyone who uses any make(1) features in /etc/mk.conf. 488 489 ---cut here--- 490 #!/bin/sh 491 . /etc/mk.conf 492 493 if [ -z $NETBSDSRCDIR ] ; then 494 NETBSDSRCDIR=/usr/src 495 fi 496 if [ \! -d $NETBSDSRCDIR ] ; then 497 echo Unable to find sources 498 exit 1 499 fi 500 find $NETBSDSRCDIR -name \*.o -o -name obj.\* -o -name obj -exec rm \{\} \; 501 502 if [ -z $BSDOBJDIR ] ; then 503 BSDOBJDIR=/usr/obj 504 fi 505 if [ -d $BSDOBJDIR ] ; then 506 rm -rf $BSDOBJDIR 507 fi 508 509 cd $NETBSDSRCDIR && make cleandir 510 511 ---cut here--- 512 513 Critical utilities: 514 ^^^^^^^^^^^^^^^^^^^ 515 usr.bin/make 516 usr.bin/yacc 517 usr.bin/lex 518 usr.bin/xlint 519 usr.bin/config 520 521 Other problems and possible solutions: 522 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 523 Symptom:Complaints involving a Makefile. 524 Fix: Rebuild usr.bin/make: 525 cd usr.bin/make && make && make install 526 Or, a failsafe method if that doesn't work: 527 cd usr.bin/make && cc -DMAKE_NATIVE *.c -I. -o make \ 528 && install make /usr/bin 529 530 Fix: Make sure .mk files are up to date. 531 cd share/mk && make install 532 533 Symptom:Kernel `config' fails to configure any kernel, including GENERIC. 534 Fix: Rebuild usr.bin/config 535 536 Symptom:Obsolete intermediate files are used during compilation 537 Fix: Try the following sequence of commands in the directory in question. 538 make cleandir; rm `make print-objdir`; make cleandir; make obj 539 (If you built the tree without "make obj" in the past, obsolete files 540 may remain. The command tries to clean everything up) 541 542 Symptom:.../sysinst/run.c:xx: warning: initialization from incompatible 543 pointer type 544 Fix: Rebuild and install usr.bin/menuc 545 546 Symptom:mklocale not found during build in share/locale/ctype 547 Fix: Build and install usr.bin/mklocale 548 549 Symptom:undefined reference to `__assert13' or `__unsetenv13' 550 Fix: Rebuild and install lib/libc 551 552 Symptom:usr.bin/config fails to build. 553 Fix: Try building with -DMAKE_BOOTSTRAP added to CFLAGS in Makefile. 554 555 Symptom:undefined reference to `getprogname' or `setprogname' 556 Fix: Rebuild and install lib/libc 557 558 Symptom:Update build fails in src/tools/gcc complaining that a variable 559 (e.g. CPPFLAGS) has changed since the previous run. 560 Fix: Run "nbmake-${ARCH} clean" in src/tools/gcc or do a clean build. 561 562 Symptom:cvs [update aborted]: cannot open directory /cvsroot/src/...: 563 No such file or directory. 564 Cause: If a directory is created by mistake, then it is sometimes 565 deleted from the CVS repository using administrative commands 566 that bypass the normal cvs access controls. If your cvs working tree 567 contains references to a directory that has been deleted on the 568 server in this way, then "cvs update" reports this error. 569 Fix: Recursively delete the affected directory from your working tree 570 and try the update again. 571
Indexes created Sun Oct 12 09:09:55 GMT 2025