UPDATING revision 1.368
1 $NetBSD: UPDATING,v 1.368 2024/11/22 14:53:12 riastradh Exp $ 2 3 This file (UPDATING) is intended to be a brief reference to recent 4 changes that might cause problems in the build process, and a guide for 5 what to do if something doesn't work. 6 7 For a more detailed description of the recommended way to build NetBSD 8 using build.sh, see the BUILDING file. 9 10 Note that much of the advice in this UPDATING file was written before 11 build.sh existed. Nevertheless, the advice here may be useful for 12 working around specific problems with build.sh. 13 14 Sections are marked with "^^^^^". After the section on "Recent changes" 15 are several sections containing more general information. 16 17 See also: BUILDING, build.sh, Makefile. 18 19 Recent changes: 20 ^^^^^^^^^^^^^^^ 21 22 20241122: 23 Changes to libarchive's build options require cleaning its 24 objdir in update builds: 25 26 cd src/external/bsd/libarchive && $TOOLDIR/bin/nbmake-$ARCH cleandir 27 28 Changes to bsd.lib.mk to improve detecting differences in 29 shared library symbol exports may cause some .actsym files in 30 the objdir to become stale in update builds. 31 32 To work around this, delete all files matching `*.actsym' in 33 the objdir. For example: 34 35 find OBJDIR -name '*.actsym' -type f -print0 | xargs -0 rm 36 37 20241104: 38 Changes around the integration of the zstd compression 39 library may require manual deletion of some object directories 40 if you update from a build from a few days ago: 41 rm -rf DESTDIR/usr/lib/*/libarchive* 42 rm -rf DESTDIR/usr/lib/*/libmagic* 43 rm -rf DESTDIR/usr/lib/*/libzstd* 44 rm -rf DESTDIR/usr/lib/libarchive* 45 rm -rf DESTDIR/usr/lib/libmagic* 46 rm -rf DESTDIR/usr/lib/libzstd* 47 rm -rf OBJDIR/compat/*/*/external/bsd/file/ 48 rm -rf OBJDIR/compat/*/*/external/bsd/libarchive/ 49 rm -rf OBJDIR/compat/*/*/external/bsd/zstd/ 50 rm -rf OBJDIR/external/bsd/file/ 51 rm -rf OBJDIR/external/bsd/libarchive/ 52 rm -rf OBJDIR/external/bsd/zstd/ 53 rm -rf OBJDIR/external/gpl3/gcc/ 54 rm -rf OBJDIR/tools/gcc/ 55 56 20240926: 57 Building -current when running on amd64 from about 20240924 58 to about 20240926 will not work due to a "malloc_np.h" 59 error. Please downgrade to a version from 20240924 or older, 60 or upgrade to a newer version e.g. using a binary release from 61 the NetBSD build servers. 62 63 20240923: 64 Update builds will require cleaning the object directories for 65 jemalloc (like: "rm -rf $OBJ/external/bsd/jemalloc*") and the 66 libc dependency files (like: "rm $OBJ/lib/libc/jemalloc.*"). 67 On architectures with COMPAT libs, those also have to be cleaned, 68 e.g. on sparc64: "rm -rf $OBJ/compat/sparc64/sparc/external/bsd/jemalloc*" 69 and "rm $OBJ/compat/sparc64/sparc/lib/libc/jemalloc.*". 70 71 20240817: 72 Update builds will require cleaning in src/external/historical/nawk/bin 73 in order to remove the awk binary since now there is a subdirectory 74 called awk. 75 76 20240812: 77 Update builds will require removing the tools objdir for gdb 78 if building with MKCROSSGDB=yes. All architectures use gdb.old 79 now. 80 81 20240630: 82 Update builds will require removing the tools objdir for binutils 83 (as most architectures should use binutils.old now). 84 85 20240610: 86 Update builds are likely to require removing the objdir for 87 distrib/<arch>/cdroms - or at least references to libarchive 88 from within there after the recent libarchive update. 89 90 20240504: 91 Update builds on x86 require making the `cleandir' target in 92 lib/libm, or deleting the objdir for lib/libm. 93 94 20240423: 95 Update builds with x11 may fail after switching to Mesa 21. 96 97 20240410: 98 Update builds may require manually deleting 99 $DESTDIR/var/run/named (and, potentially, 100 $DESTDIR/var/run/lwresd) in order to avoid checkflist failure. 101 102 20240409: 103 32-bit compatibility libraries were moved into the base32 104 and debug32 sets. HTML man pages were moved into the manhtml set. 105 106 20230828: 107 If: 108 - you updated to current and ran postinstall between 20230826 109 and 20230828 inclusive, and 110 - you are not using anything outside the base system to 111 populate /etc/openssl/certs (e.g., manually adding 112 certificates or using ca-certificates or mozilla-rootcerts or 113 mozilla-rootcerts-openssl from pkgsrc), 114 delete /etc/openssl/certs and rerun postinstall or just 115 `certctl rehash'. 116 117 Otherwise, certctl(8) will think someone else has left 118 /etc/openssl/certs in the way and will avoid clobbering it on 119 upgrade from older NetBSD. 120 121 20230718: 122 build.sh will use mk.conf in the same directory as build.sh instead 123 of /etc/mk.conf, unless MAKECONF is already defined in the environment. 124 125 20230714: 126 Import of gcc 10.5 requires a clean build of at least tools/gcc 127 and external/gpl3/gcc* object directories. An update build 128 without cleaning will result in obscure failures in rare corner 129 cases. 130 131 20230604: 132 Building native kernels with the installed make(1) instead of 133 the $TOOLDIR/bin/nbmake-${arch} wrapper will fail if userland 134 has not been updated. Note that this is an unsupported build 135 configuration (but usually works if userland is -current enough). 136 137 20230523: 138 ctfmerge was fixed as macOS host tool. This may require a clean tools 139 rebuild to detect the update to tools/compat/configure. 140 141 20230515: 142 New openssl build requires removal of the crypto/external/bsd/openssl 143 and tests/crypto/libcrypto subdirectories in the ${OBJDIR} hierarchy. 144 Otherwise test results may be wrong due to wrong "golden" output 145 files being installed. 146 147 20230322: 148 Fixes for moved /lib/libisns* mean that builds will complain about 149 ./usr/lib/libisns* being missing until src/lib/libisns is forced 150 to (re)install the files, including the symlinks. The simplest 151 way is to clean this subdir before building. 152 153 20230112: 154 New binutils require an updated ld.elf_so. If you are doing 155 (unsupported) in-place self builds (with the -E flag to build.sh), 156 make sure to have installed latest ld.elf_so before you rebuild 157 userland. 158 159 20221116: 160 The addition to NetBSD's version of UFS2 of support for extended 161 attributes broke backward compatibility with previous releases 162 of NetBSD, so UFS2 has been restored to being compatible with 163 previous NetBSD releases by disabling extended attributes. 164 (Note that ACLs are implemented as extended attributes, so 165 this changes disables ACLs as well.) 166 167 Support for UFS2 with extended attributes is now available in a new 168 UFS variant called UFS2ea. If you have created extended attributes 169 in an original UFS2 file system then "fsck -p" will now fail due to 170 the unexpected presence of extended attributes and "fsck -y" will 171 remove all extended attributes. If you wish to preserve extended 172 attributes rather than delete them, there is a utility to convert 173 a UFS2 file system to UFS2ea and leave extended attributes in place, 174 but this should be used with caution since it will preserve any 175 extended attributes that have been corrupted by the backward 176 incompatibility too. 177 178 If you wish to use a UFS2ea file system as your root file system, 179 then you will need to update your boot loader to a version that 180 supports UFS2ea. 181 182 For more information, see: 183 https://wiki.netbsd.org/features/UFS2ea 184 185 20221111: 186 The new libdrm import worsened the conflict issues for the 187 kdump/ktruss ioctl, and i915 now conflicts with base, and has 188 been turned off. This will cause update build issues like: 189 190 kdump-ioctl.c:12175:143: error: 'DRM_IOCTL_I915_DESTROY_HEAP' 191 undeclared here (not in a function); 192 did you mean 'DRM_IOCTL_MODE_DESTROY_DUMB'? 193 194 You'll need to clean usr.bin/ktruss, usr.bin/kdump, and rescue. 195 196 20220921: 197 Since the kernel version was bumped to 9.99.100, the bootloader 198 must be updated on x86 or EFI boot platforms in order to load 199 modules from boot.cfg(5) or interactively at the bootloader 200 prompt. 201 202 The kernel can still load modules at runtime, with modload(8) 203 or modules.conf(5), with no bootloader update. This will not 204 affect release branches because it only applies to patch 205 numbers >=100. 206 207 On x86 (i386 or amd64) with BIOS boot, this requires copying a 208 new /usr/mdec/boot to /boot. You can build this in the destdir 209 by running `$TOOLDIR/bin/nbmake-$ARCH dependall install' in 210 sys/arch/i386/stand. 211 212 On platforms with EFI boot (including x86 with EFI boot), this 213 requires copying a new /usr/mdec/boot*.efi to the EFI/BOOT/ 214 directory on the EFI system partition. The boot*.efi file is: 215 216 aarch64 bootaa64.efi 217 amd64 bootx64.efi 218 arm bootarm.efi (32-bit arm) 219 i386 bootx32.efi 220 221 You can build this in the destdir by running 222 `$TOOLDIR/bin/nbmake-$ARCH dependall install' in 223 sys/stand/efiboot/boot*. 224 225 20220821: 226 Support for building extsrc/ has been deprecated. 227 EXTSRCSRCDIR and MKEXTSRC have been deprecated. 228 229 20220714: 230 Updates to xorg-server and associated drivers may cause builds to 231 fail. Cleaning both DESTDIR and the xorg build tree may be needed. 232 233 20220628: 234 Changes in the build of src/games/robots require a clean build 235 of that program. 236 237 20211116: 238 Changes in the xorg build require clean obj dirs for external/mit/xorg 239 (or a full clean/non-update build). 240 241 20211112: 242 Device tree sources were updated for evbarm. Some device nodes 243 (in particular, ld(4) devices on the ROCKPro64) will be renumbered. 244 245 20211014: 246 MKPIE default for aarch64 has changed, a clean build is required. 247 248 20210917: 249 Changed MNT_ACLS to default to POSIX1e ACLs instead of NFSv4 250 ACLs, to be compatible with FreeBSD. If you are using NFSv4 ACLs 251 and have "acls" in /etc/fstab you'll need to change it to "nfs4acls". 252 253 20210711: 254 Updated GMP sources may require cleaning in tools/gmp and/or in 255 external/lgpl3/gmp/lib, particularly if GCC itself does not build. 256 257 20210417: 258 GCC 10 was enabled for several platforms. If builds fail in either 259 tools/gcc or external/gpl3/gcc, first try cleaning those objects and 260 removing the $DESTDIR/usr/include/g++ subdirectory. 261 262 20201016: 263 MIPS kernel modules have been disabled until they work. This will 264 turn up in extra files in the DESTDIR, which should be cleaned. 265 266 20200925: 267 GNU MPC and MPFR have been updated. At least MPFR needs cleaning 268 in both the tools and external dirs. 269 270 20200912: 271 GCC 9 has arrived for Arm and x86 platforms, and will be coming 272 for several more, as will binutils 2.34. Clean as required. 273 274 20200907: 275 GCC 9 is coming, and binutils has been updated for MIPS. This 276 probably requires cleaning the tools/binutils. 277 278 20200811: 279 GCC updates may require cleaning the objdir. This may occur 280 due to GCC 7.5 update, GCC 8.4 move to gcc.old, or the upcoming 281 GCC 9 upgrade. 282 283 20200614: 284 blacklist* has been renamed to blocklist*. postinstall(8) 285 should handle the migration. 286 287 20200601: 288 Due to a mistake in LIBISPRIVATE handling, .so libraries were 289 created in the build directories and need cleaning. 290 291 20200311: 292 GCC 8 ports will need cleaning in src/tools/gcc and 293 src/external/gpl3/gcc due to GCC 8.4 update. 294 295 20191118: 296 More architectures were switched to gcc8: 297 i386, ia64 powerpc64, sparc, sparc64, arm 298 The same comments as in 20191022 apply. 299 300 20191112: 301 The LLVM update requires a clean rebuild for all architectures using 302 LLVM during the tools build phase (i386, amd64, aarch64). 303 304 20191022: 305 Ports amd64 and aarch64 switched to GCC 8.3 by default. 306 In-place ("expert mode", build.sh -E) builds are not supported 307 when going from a GCC 7 userland to GCC 8. Do a regular 308 build to a different DESTDIR (or preferably: build.sh -U) at least 309 once and install sets, or download comp.{tar.xz,tgz} from the 310 daily builds and install that before doing the next in-place build. 311 312 20191001: 313 GCC 8.3 was imported. Builds of src/tools/gcc may fail if 314 old builds with GCC 7 output now uses GCC 8. Clean this 315 directory, and also clean src/external/gpl3/gcc. 316 317 20190903: 318 Files with names that coincide with existing files' names on 319 case-insensitive file systems were inadvertently committed, for 320 radeon GPU firmware. We cannot mark these as obsolete for 321 postinstall to fix, so if you updated src since 2019-08-26, and 322 ran build.sh distribution or ran build.sh release, you must 323 manually delete the following files in your DESTDIR (which is 324 usually $OBJDIR/destir.$ARCH), or from / if you have installed 325 them: 326 327 /libdata/firmware/radeon/bonaire_ce.bin 328 /libdata/firmware/radeon/bonaire_mc.bin 329 /libdata/firmware/radeon/bonaire_me.bin 330 /libdata/firmware/radeon/bonaire_mec.bin 331 /libdata/firmware/radeon/bonaire_pfp.bin 332 /libdata/firmware/radeon/bonaire_rlc.bin 333 /libdata/firmware/radeon/bonaire_sdma.bin 334 /libdata/firmware/radeon/bonaire_smc.bin 335 /libdata/firmware/radeon/bonaire_uvd.bin 336 /libdata/firmware/radeon/hainan_ce.bin 337 /libdata/firmware/radeon/hainan_mc.bin 338 /libdata/firmware/radeon/hainan_me.bin 339 /libdata/firmware/radeon/hainan_pfp.bin 340 /libdata/firmware/radeon/hainan_rlc.bin 341 /libdata/firmware/radeon/hainan_smc.bin 342 /libdata/firmware/radeon/hawaii_ce.bin 343 /libdata/firmware/radeon/hawaii_mc.bin 344 /libdata/firmware/radeon/hawaii_me.bin 345 /libdata/firmware/radeon/hawaii_mec.bin 346 /libdata/firmware/radeon/hawaii_pfp.bin 347 /libdata/firmware/radeon/hawaii_rlc.bin 348 /libdata/firmware/radeon/hawaii_sdma.bin 349 /libdata/firmware/radeon/hawaii_smc.bin 350 /libdata/firmware/radeon/kabini_ce.bin 351 /libdata/firmware/radeon/kabini_me.bin 352 /libdata/firmware/radeon/kabini_mec.bin 353 /libdata/firmware/radeon/kabini_pfp.bin 354 /libdata/firmware/radeon/kabini_rlc.bin 355 /libdata/firmware/radeon/kabini_sdma.bin 356 /libdata/firmware/radeon/kaveri_ce.bin 357 /libdata/firmware/radeon/kaveri_me.bin 358 /libdata/firmware/radeon/kaveri_mec.bin 359 /libdata/firmware/radeon/kaveri_mec2.bin 360 /libdata/firmware/radeon/kaveri_pfp.bin 361 /libdata/firmware/radeon/kaveri_rlc.bin 362 /libdata/firmware/radeon/kaveri_sdma.bin 363 /libdata/firmware/radeon/mullins_ce.bin 364 /libdata/firmware/radeon/mullins_me.bin 365 /libdata/firmware/radeon/mullins_mec.bin 366 /libdata/firmware/radeon/mullins_pfp.bin 367 /libdata/firmware/radeon/mullins_rlc.bin 368 /libdata/firmware/radeon/mullins_sdma.bin 369 /libdata/firmware/radeon/oland_ce.bin 370 /libdata/firmware/radeon/oland_mc.bin 371 /libdata/firmware/radeon/oland_me.bin 372 /libdata/firmware/radeon/oland_pfp.bin 373 /libdata/firmware/radeon/oland_rlc.bin 374 /libdata/firmware/radeon/oland_smc.bin 375 /libdata/firmware/radeon/pitcairn_ce.bin 376 /libdata/firmware/radeon/pitcairn_mc.bin 377 /libdata/firmware/radeon/pitcairn_me.bin 378 /libdata/firmware/radeon/pitcairn_pfp.bin 379 /libdata/firmware/radeon/pitcairn_rlc.bin 380 /libdata/firmware/radeon/pitcairn_smc.bin 381 /libdata/firmware/radeon/tahiti_ce.bin 382 /libdata/firmware/radeon/tahiti_mc.bin 383 /libdata/firmware/radeon/tahiti_me.bin 384 /libdata/firmware/radeon/tahiti_pfp.bin 385 /libdata/firmware/radeon/tahiti_rlc.bin 386 /libdata/firmware/radeon/tahiti_smc.bin 387 /libdata/firmware/radeon/verde_ce.bin 388 /libdata/firmware/radeon/verde_mc.bin 389 /libdata/firmware/radeon/verde_me.bin 390 /libdata/firmware/radeon/verde_pfp.bin 391 /libdata/firmware/radeon/verde_rlc.bin 392 /libdata/firmware/radeon/verde_smc.bin 393 394 We will re-import these radeon firmware images another way 395 later. 396 397 20190727: 398 The uefi bootloader has gained tftp support and needs a clean 399 build. If you do update builds, manually clean its object 400 directory by something like: 401 cd sys/arch/i386/stand/efiboot && make clean 402 403 20190723: 404 The jemalloc allocator in libc is now built without extended 405 debugging (for performance reasons). In update builds make sure 406 to rebuild it completely, by removing all affected object files, 407 including compat builds, something like: 408 cd /usr/obj && find . -type d -name jemalloc|xargs rm -rf 409 410 20190207: 411 GCC 7 switched for many ports. Update builds are likely to fail. 412 413 Hints for a more successful build: 414 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 415 Use build.sh, but do not use its "expert mode": 416 This will automatically build the tools in the 417 correct order, and it will keep the tools and the 418 new build products from interfering with the running 419 system. This will allow you to ignore most of the 420 other advice in this file. 421 Build a new kernel first: 422 This makes sure that any new system calls or features 423 expected by the new userland will be present. This 424 helps to avoid critical errors when upgrading. 425 Use object directories: 426 This helps to keep stale object 427 files from polluting the build if a Makefile "forgets" 428 about one. It also makes it easier to clean up after 429 a build. It's also necessary if you want to use the 430 same source tree for multiple machines. 431 To use object directories with build.sh: 432 a) invoke build.sh with the "-M" or "-O" options. 433 To use object directories without using build.sh: 434 a) cd /usr/src ; make cleandir 435 b) Add "OBJMACHINE=yes" to /etc/mk.conf 436 c) Add "MKOBJDIRS=yes" to /etc/mk.conf 437 d) cd /usr/src ; make build 438 Note that running "make obj" in a directory will create 439 in obj.$MACHINE directory. 440 Build to a DESTDIR: 441 This helps to keep old installed files (especially libraries) 442 from interfering with the new build. 443 To build to a DESTDIR with build.sh, use the "-D" option. 444 To build to a DESTDIR without using build.sh, set the DESTDIR 445 environment variable before running make build. It should be 446 set to the pathname of an initially empty directory. 447 Problems: if you do not use build.sh, you might need to 448 update critical utilities without using DESTDIR since 449 nothing is executed from what is installed in DESTDIR. 450 (See critical utils, below.) 451 Build often: 452 This keeps critical utilities current enough to not choke 453 on any other part of the source tree that depends on up to 454 date functionality. If you use build.sh, you should not have 455 this problem. 456 457 What to do if things don't work: 458 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 459 When things don't work there are usually a few things that commonly 460 should be done. 461 1) make includes 462 This should be done automatically by make build. 463 2) cd share/mk && make install 464 Again, automatically done by make build. 465 466 Failsafe rebuild of a small part of the tree: 467 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 468 To make sure you rebuild something correctly you want to do 469 something like the following: 470 1) Make sure the includes and .mk files are up to date. 471 2) Make sure any program used to build the particular 472 utility is up to date. (yacc, lex, etc...) 473 3) cd ...path/to/util... 474 make cleandir 475 rm ...all obj directories... 476 make cleandir # yes, again 477 make obj 478 make depend && make 479 480 Failsafe rebuild of the entire tree: 481 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 482 If you really want to make sure the source tree is clean and 483 ready for a build try the following. Note that sourcing /etc/mk.conf 484 (a make(1) Makefile) in this manner is not right, and will not work 485 for anyone who uses any make(1) features in /etc/mk.conf. 486 487 ---cut here--- 488 #!/bin/sh 489 . /etc/mk.conf 490 491 if [ -z $NETBSDSRCDIR ] ; then 492 NETBSDSRCDIR=/usr/src 493 fi 494 if [ \! -d $NETBSDSRCDIR ] ; then 495 echo Unable to find sources 496 exit 1 497 fi 498 find $NETBSDSRCDIR -name \*.o -o -name obj.\* -o -name obj -exec rm \{\} \; 499 500 if [ -z $BSDOBJDIR ] ; then 501 BSDOBJDIR=/usr/obj 502 fi 503 if [ -d $BSDOBJDIR ] ; then 504 rm -rf $BSDOBJDIR 505 fi 506 507 cd $NETBSDSRCDIR && make cleandir 508 509 ---cut here--- 510 511 Critical utilities: 512 ^^^^^^^^^^^^^^^^^^^ 513 usr.bin/make 514 usr.bin/yacc 515 usr.bin/lex 516 usr.bin/xlint 517 usr.bin/config 518 519 Other problems and possible solutions: 520 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 521 Symptom:Complaints involving a Makefile. 522 Fix: Rebuild usr.bin/make: 523 cd usr.bin/make && make && make install 524 Or, a failsafe method if that doesn't work: 525 cd usr.bin/make && cc -DMAKE_NATIVE *.c -I. -o make \ 526 && install make /usr/bin 527 528 Fix: Make sure .mk files are up to date. 529 cd share/mk && make install 530 531 Symptom:Kernel `config' fails to configure any kernel, including GENERIC. 532 Fix: Rebuild usr.bin/config 533 534 Symptom:Obsolete intermediate files are used during compilation 535 Fix: Try the following sequence of commands in the directory in question. 536 make cleandir; rm `make print-objdir`; make cleandir; make obj 537 (If you built the tree without "make obj" in the past, obsolete files 538 may remain. The command tries to clean everything up) 539 540 Symptom:.../sysinst/run.c:xx: warning: initialization from incompatible 541 pointer type 542 Fix: Rebuild and install usr.bin/menuc 543 544 Symptom:mklocale not found during build in share/locale/ctype 545 Fix: Build and install usr.bin/mklocale 546 547 Symptom:undefined reference to `__assert13' or `__unsetenv13' 548 Fix: Rebuild and install lib/libc 549 550 Symptom:usr.bin/config fails to build. 551 Fix: Try building with -DMAKE_BOOTSTRAP added to CFLAGS in Makefile. 552 553 Symptom:undefined reference to `getprogname' or `setprogname' 554 Fix: Rebuild and install lib/libc 555 556 Symptom:Update build fails in src/tools/gcc complaining that a variable 557 (e.g. CPPFLAGS) has changed since the previous run. 558 Fix: Run "nbmake-${ARCH} clean" in src/tools/gcc or do a clean build. 559 560 Symptom:cvs [update aborted]: cannot open directory /cvsroot/src/...: 561 No such file or directory. 562 Cause: If a directory is created by mistake, then it is sometimes 563 deleted from the CVS repository using administrative commands 564 that bypass the normal cvs access controls. If your cvs working tree 565 contains references to a directory that has been deleted on the 566 server in this way, then "cvs update" reports this error. 567 Fix: Recursively delete the affected directory from your working tree 568 and try the update again. 569
Indexes created Sun Oct 12 09:09:55 GMT 2025