UPDATING revision 1.367
1 $NetBSD: UPDATING,v 1.367 2024/11/22 14:30:28 riastradh Exp $ 2 3 This file (UPDATING) is intended to be a brief reference to recent 4 changes that might cause problems in the build process, and a guide for 5 what to do if something doesn't work. 6 7 For a more detailed description of the recommended way to build NetBSD 8 using build.sh, see the BUILDING file. 9 10 Note that much of the advice in this UPDATING file was written before 11 build.sh existed. Nevertheless, the advice here may be useful for 12 working around specific problems with build.sh. 13 14 Sections are marked with "^^^^^". After the section on "Recent changes" 15 are several sections containing more general information. 16 17 See also: BUILDING, build.sh, Makefile. 18 19 Recent changes: 20 ^^^^^^^^^^^^^^^ 21 22 20241122: 23 Changes to bsd.lib.mk to improve detecting differences in 24 shared library symbol exports may cause some .actsym files in 25 the objdir to become stale in update builds. 26 27 To work around this, delete all files matching `*.actsym' in 28 the objdir. For example: 29 30 find OBJDIR -name '*.actsym' -type f -print0 | xargs -0 rm 31 32 20241104: 33 Changes around the integration of the zstd compression 34 library may require manual deletion of some object directories 35 if you update from a build from a few days ago: 36 rm -rf DESTDIR/usr/lib/*/libarchive* 37 rm -rf DESTDIR/usr/lib/*/libmagic* 38 rm -rf DESTDIR/usr/lib/*/libzstd* 39 rm -rf DESTDIR/usr/lib/libarchive* 40 rm -rf DESTDIR/usr/lib/libmagic* 41 rm -rf DESTDIR/usr/lib/libzstd* 42 rm -rf OBJDIR/compat/*/*/external/bsd/file/ 43 rm -rf OBJDIR/compat/*/*/external/bsd/libarchive/ 44 rm -rf OBJDIR/compat/*/*/external/bsd/zstd/ 45 rm -rf OBJDIR/external/bsd/file/ 46 rm -rf OBJDIR/external/bsd/libarchive/ 47 rm -rf OBJDIR/external/bsd/zstd/ 48 rm -rf OBJDIR/external/gpl3/gcc/ 49 rm -rf OBJDIR/tools/gcc/ 50 51 20240926: 52 Building -current when running on amd64 from about 20240924 53 to about 20240926 will not work due to a "malloc_np.h" 54 error. Please downgrade to a version from 20240924 or older, 55 or upgrade to a newer version e.g. using a binary release from 56 the NetBSD build servers. 57 58 20240923: 59 Update builds will require cleaning the object directories for 60 jemalloc (like: "rm -rf $OBJ/external/bsd/jemalloc*") and the 61 libc dependency files (like: "rm $OBJ/lib/libc/jemalloc.*"). 62 On architectures with COMPAT libs, those also have to be cleaned, 63 e.g. on sparc64: "rm -rf $OBJ/compat/sparc64/sparc/external/bsd/jemalloc*" 64 and "rm $OBJ/compat/sparc64/sparc/lib/libc/jemalloc.*". 65 66 20240817: 67 Update builds will require cleaning in src/external/historical/nawk/bin 68 in order to remove the awk binary since now there is a subdirectory 69 called awk. 70 71 20240812: 72 Update builds will require removing the tools objdir for gdb 73 if building with MKCROSSGDB=yes. All architectures use gdb.old 74 now. 75 76 20240630: 77 Update builds will require removing the tools objdir for binutils 78 (as most architectures should use binutils.old now). 79 80 20240610: 81 Update builds are likely to require removing the objdir for 82 distrib/<arch>/cdroms - or at least references to libarchive 83 from within there after the recent libarchive update. 84 85 20240504: 86 Update builds on x86 require making the `cleandir' target in 87 lib/libm, or deleting the objdir for lib/libm. 88 89 20240423: 90 Update builds with x11 may fail after switching to Mesa 21. 91 92 20240410: 93 Update builds may require manually deleting 94 $DESTDIR/var/run/named (and, potentially, 95 $DESTDIR/var/run/lwresd) in order to avoid checkflist failure. 96 97 20240409: 98 32-bit compatibility libraries were moved into the base32 99 and debug32 sets. HTML man pages were moved into the manhtml set. 100 101 20230828: 102 If: 103 - you updated to current and ran postinstall between 20230826 104 and 20230828 inclusive, and 105 - you are not using anything outside the base system to 106 populate /etc/openssl/certs (e.g., manually adding 107 certificates or using ca-certificates or mozilla-rootcerts or 108 mozilla-rootcerts-openssl from pkgsrc), 109 delete /etc/openssl/certs and rerun postinstall or just 110 `certctl rehash'. 111 112 Otherwise, certctl(8) will think someone else has left 113 /etc/openssl/certs in the way and will avoid clobbering it on 114 upgrade from older NetBSD. 115 116 20230718: 117 build.sh will use mk.conf in the same directory as build.sh instead 118 of /etc/mk.conf, unless MAKECONF is already defined in the environment. 119 120 20230714: 121 Import of gcc 10.5 requires a clean build of at least tools/gcc 122 and external/gpl3/gcc* object directories. An update build 123 without cleaning will result in obscure failures in rare corner 124 cases. 125 126 20230604: 127 Building native kernels with the installed make(1) instead of 128 the $TOOLDIR/bin/nbmake-${arch} wrapper will fail if userland 129 has not been updated. Note that this is an unsupported build 130 configuration (but usually works if userland is -current enough). 131 132 20230523: 133 ctfmerge was fixed as macOS host tool. This may require a clean tools 134 rebuild to detect the update to tools/compat/configure. 135 136 20230515: 137 New openssl build requires removal of the crypto/external/bsd/openssl 138 and tests/crypto/libcrypto subdirectories in the ${OBJDIR} hierarchy. 139 Otherwise test results may be wrong due to wrong "golden" output 140 files being installed. 141 142 20230322: 143 Fixes for moved /lib/libisns* mean that builds will complain about 144 ./usr/lib/libisns* being missing until src/lib/libisns is forced 145 to (re)install the files, including the symlinks. The simplest 146 way is to clean this subdir before building. 147 148 20230112: 149 New binutils require an updated ld.elf_so. If you are doing 150 (unsupported) in-place self builds (with the -E flag to build.sh), 151 make sure to have installed latest ld.elf_so before you rebuild 152 userland. 153 154 20221116: 155 The addition to NetBSD's version of UFS2 of support for extended 156 attributes broke backward compatibility with previous releases 157 of NetBSD, so UFS2 has been restored to being compatible with 158 previous NetBSD releases by disabling extended attributes. 159 (Note that ACLs are implemented as extended attributes, so 160 this changes disables ACLs as well.) 161 162 Support for UFS2 with extended attributes is now available in a new 163 UFS variant called UFS2ea. If you have created extended attributes 164 in an original UFS2 file system then "fsck -p" will now fail due to 165 the unexpected presence of extended attributes and "fsck -y" will 166 remove all extended attributes. If you wish to preserve extended 167 attributes rather than delete them, there is a utility to convert 168 a UFS2 file system to UFS2ea and leave extended attributes in place, 169 but this should be used with caution since it will preserve any 170 extended attributes that have been corrupted by the backward 171 incompatibility too. 172 173 If you wish to use a UFS2ea file system as your root file system, 174 then you will need to update your boot loader to a version that 175 supports UFS2ea. 176 177 For more information, see: 178 https://wiki.netbsd.org/features/UFS2ea 179 180 20221111: 181 The new libdrm import worsened the conflict issues for the 182 kdump/ktruss ioctl, and i915 now conflicts with base, and has 183 been turned off. This will cause update build issues like: 184 185 kdump-ioctl.c:12175:143: error: 'DRM_IOCTL_I915_DESTROY_HEAP' 186 undeclared here (not in a function); 187 did you mean 'DRM_IOCTL_MODE_DESTROY_DUMB'? 188 189 You'll need to clean usr.bin/ktruss, usr.bin/kdump, and rescue. 190 191 20220921: 192 Since the kernel version was bumped to 9.99.100, the bootloader 193 must be updated on x86 or EFI boot platforms in order to load 194 modules from boot.cfg(5) or interactively at the bootloader 195 prompt. 196 197 The kernel can still load modules at runtime, with modload(8) 198 or modules.conf(5), with no bootloader update. This will not 199 affect release branches because it only applies to patch 200 numbers >=100. 201 202 On x86 (i386 or amd64) with BIOS boot, this requires copying a 203 new /usr/mdec/boot to /boot. You can build this in the destdir 204 by running `$TOOLDIR/bin/nbmake-$ARCH dependall install' in 205 sys/arch/i386/stand. 206 207 On platforms with EFI boot (including x86 with EFI boot), this 208 requires copying a new /usr/mdec/boot*.efi to the EFI/BOOT/ 209 directory on the EFI system partition. The boot*.efi file is: 210 211 aarch64 bootaa64.efi 212 amd64 bootx64.efi 213 arm bootarm.efi (32-bit arm) 214 i386 bootx32.efi 215 216 You can build this in the destdir by running 217 `$TOOLDIR/bin/nbmake-$ARCH dependall install' in 218 sys/stand/efiboot/boot*. 219 220 20220821: 221 Support for building extsrc/ has been deprecated. 222 EXTSRCSRCDIR and MKEXTSRC have been deprecated. 223 224 20220714: 225 Updates to xorg-server and associated drivers may cause builds to 226 fail. Cleaning both DESTDIR and the xorg build tree may be needed. 227 228 20220628: 229 Changes in the build of src/games/robots require a clean build 230 of that program. 231 232 20211116: 233 Changes in the xorg build require clean obj dirs for external/mit/xorg 234 (or a full clean/non-update build). 235 236 20211112: 237 Device tree sources were updated for evbarm. Some device nodes 238 (in particular, ld(4) devices on the ROCKPro64) will be renumbered. 239 240 20211014: 241 MKPIE default for aarch64 has changed, a clean build is required. 242 243 20210917: 244 Changed MNT_ACLS to default to POSIX1e ACLs instead of NFSv4 245 ACLs, to be compatible with FreeBSD. If you are using NFSv4 ACLs 246 and have "acls" in /etc/fstab you'll need to change it to "nfs4acls". 247 248 20210711: 249 Updated GMP sources may require cleaning in tools/gmp and/or in 250 external/lgpl3/gmp/lib, particularly if GCC itself does not build. 251 252 20210417: 253 GCC 10 was enabled for several platforms. If builds fail in either 254 tools/gcc or external/gpl3/gcc, first try cleaning those objects and 255 removing the $DESTDIR/usr/include/g++ subdirectory. 256 257 20201016: 258 MIPS kernel modules have been disabled until they work. This will 259 turn up in extra files in the DESTDIR, which should be cleaned. 260 261 20200925: 262 GNU MPC and MPFR have been updated. At least MPFR needs cleaning 263 in both the tools and external dirs. 264 265 20200912: 266 GCC 9 has arrived for Arm and x86 platforms, and will be coming 267 for several more, as will binutils 2.34. Clean as required. 268 269 20200907: 270 GCC 9 is coming, and binutils has been updated for MIPS. This 271 probably requires cleaning the tools/binutils. 272 273 20200811: 274 GCC updates may require cleaning the objdir. This may occur 275 due to GCC 7.5 update, GCC 8.4 move to gcc.old, or the upcoming 276 GCC 9 upgrade. 277 278 20200614: 279 blacklist* has been renamed to blocklist*. postinstall(8) 280 should handle the migration. 281 282 20200601: 283 Due to a mistake in LIBISPRIVATE handling, .so libraries were 284 created in the build directories and need cleaning. 285 286 20200311: 287 GCC 8 ports will need cleaning in src/tools/gcc and 288 src/external/gpl3/gcc due to GCC 8.4 update. 289 290 20191118: 291 More architectures were switched to gcc8: 292 i386, ia64 powerpc64, sparc, sparc64, arm 293 The same comments as in 20191022 apply. 294 295 20191112: 296 The LLVM update requires a clean rebuild for all architectures using 297 LLVM during the tools build phase (i386, amd64, aarch64). 298 299 20191022: 300 Ports amd64 and aarch64 switched to GCC 8.3 by default. 301 In-place ("expert mode", build.sh -E) builds are not supported 302 when going from a GCC 7 userland to GCC 8. Do a regular 303 build to a different DESTDIR (or preferably: build.sh -U) at least 304 once and install sets, or download comp.{tar.xz,tgz} from the 305 daily builds and install that before doing the next in-place build. 306 307 20191001: 308 GCC 8.3 was imported. Builds of src/tools/gcc may fail if 309 old builds with GCC 7 output now uses GCC 8. Clean this 310 directory, and also clean src/external/gpl3/gcc. 311 312 20190903: 313 Files with names that coincide with existing files' names on 314 case-insensitive file systems were inadvertently committed, for 315 radeon GPU firmware. We cannot mark these as obsolete for 316 postinstall to fix, so if you updated src since 2019-08-26, and 317 ran build.sh distribution or ran build.sh release, you must 318 manually delete the following files in your DESTDIR (which is 319 usually $OBJDIR/destir.$ARCH), or from / if you have installed 320 them: 321 322 /libdata/firmware/radeon/bonaire_ce.bin 323 /libdata/firmware/radeon/bonaire_mc.bin 324 /libdata/firmware/radeon/bonaire_me.bin 325 /libdata/firmware/radeon/bonaire_mec.bin 326 /libdata/firmware/radeon/bonaire_pfp.bin 327 /libdata/firmware/radeon/bonaire_rlc.bin 328 /libdata/firmware/radeon/bonaire_sdma.bin 329 /libdata/firmware/radeon/bonaire_smc.bin 330 /libdata/firmware/radeon/bonaire_uvd.bin 331 /libdata/firmware/radeon/hainan_ce.bin 332 /libdata/firmware/radeon/hainan_mc.bin 333 /libdata/firmware/radeon/hainan_me.bin 334 /libdata/firmware/radeon/hainan_pfp.bin 335 /libdata/firmware/radeon/hainan_rlc.bin 336 /libdata/firmware/radeon/hainan_smc.bin 337 /libdata/firmware/radeon/hawaii_ce.bin 338 /libdata/firmware/radeon/hawaii_mc.bin 339 /libdata/firmware/radeon/hawaii_me.bin 340 /libdata/firmware/radeon/hawaii_mec.bin 341 /libdata/firmware/radeon/hawaii_pfp.bin 342 /libdata/firmware/radeon/hawaii_rlc.bin 343 /libdata/firmware/radeon/hawaii_sdma.bin 344 /libdata/firmware/radeon/hawaii_smc.bin 345 /libdata/firmware/radeon/kabini_ce.bin 346 /libdata/firmware/radeon/kabini_me.bin 347 /libdata/firmware/radeon/kabini_mec.bin 348 /libdata/firmware/radeon/kabini_pfp.bin 349 /libdata/firmware/radeon/kabini_rlc.bin 350 /libdata/firmware/radeon/kabini_sdma.bin 351 /libdata/firmware/radeon/kaveri_ce.bin 352 /libdata/firmware/radeon/kaveri_me.bin 353 /libdata/firmware/radeon/kaveri_mec.bin 354 /libdata/firmware/radeon/kaveri_mec2.bin 355 /libdata/firmware/radeon/kaveri_pfp.bin 356 /libdata/firmware/radeon/kaveri_rlc.bin 357 /libdata/firmware/radeon/kaveri_sdma.bin 358 /libdata/firmware/radeon/mullins_ce.bin 359 /libdata/firmware/radeon/mullins_me.bin 360 /libdata/firmware/radeon/mullins_mec.bin 361 /libdata/firmware/radeon/mullins_pfp.bin 362 /libdata/firmware/radeon/mullins_rlc.bin 363 /libdata/firmware/radeon/mullins_sdma.bin 364 /libdata/firmware/radeon/oland_ce.bin 365 /libdata/firmware/radeon/oland_mc.bin 366 /libdata/firmware/radeon/oland_me.bin 367 /libdata/firmware/radeon/oland_pfp.bin 368 /libdata/firmware/radeon/oland_rlc.bin 369 /libdata/firmware/radeon/oland_smc.bin 370 /libdata/firmware/radeon/pitcairn_ce.bin 371 /libdata/firmware/radeon/pitcairn_mc.bin 372 /libdata/firmware/radeon/pitcairn_me.bin 373 /libdata/firmware/radeon/pitcairn_pfp.bin 374 /libdata/firmware/radeon/pitcairn_rlc.bin 375 /libdata/firmware/radeon/pitcairn_smc.bin 376 /libdata/firmware/radeon/tahiti_ce.bin 377 /libdata/firmware/radeon/tahiti_mc.bin 378 /libdata/firmware/radeon/tahiti_me.bin 379 /libdata/firmware/radeon/tahiti_pfp.bin 380 /libdata/firmware/radeon/tahiti_rlc.bin 381 /libdata/firmware/radeon/tahiti_smc.bin 382 /libdata/firmware/radeon/verde_ce.bin 383 /libdata/firmware/radeon/verde_mc.bin 384 /libdata/firmware/radeon/verde_me.bin 385 /libdata/firmware/radeon/verde_pfp.bin 386 /libdata/firmware/radeon/verde_rlc.bin 387 /libdata/firmware/radeon/verde_smc.bin 388 389 We will re-import these radeon firmware images another way 390 later. 391 392 20190727: 393 The uefi bootloader has gained tftp support and needs a clean 394 build. If you do update builds, manually clean its object 395 directory by something like: 396 cd sys/arch/i386/stand/efiboot && make clean 397 398 20190723: 399 The jemalloc allocator in libc is now built without extended 400 debugging (for performance reasons). In update builds make sure 401 to rebuild it completely, by removing all affected object files, 402 including compat builds, something like: 403 cd /usr/obj && find . -type d -name jemalloc|xargs rm -rf 404 405 20190207: 406 GCC 7 switched for many ports. Update builds are likely to fail. 407 408 Hints for a more successful build: 409 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 410 Use build.sh, but do not use its "expert mode": 411 This will automatically build the tools in the 412 correct order, and it will keep the tools and the 413 new build products from interfering with the running 414 system. This will allow you to ignore most of the 415 other advice in this file. 416 Build a new kernel first: 417 This makes sure that any new system calls or features 418 expected by the new userland will be present. This 419 helps to avoid critical errors when upgrading. 420 Use object directories: 421 This helps to keep stale object 422 files from polluting the build if a Makefile "forgets" 423 about one. It also makes it easier to clean up after 424 a build. It's also necessary if you want to use the 425 same source tree for multiple machines. 426 To use object directories with build.sh: 427 a) invoke build.sh with the "-M" or "-O" options. 428 To use object directories without using build.sh: 429 a) cd /usr/src ; make cleandir 430 b) Add "OBJMACHINE=yes" to /etc/mk.conf 431 c) Add "MKOBJDIRS=yes" to /etc/mk.conf 432 d) cd /usr/src ; make build 433 Note that running "make obj" in a directory will create 434 in obj.$MACHINE directory. 435 Build to a DESTDIR: 436 This helps to keep old installed files (especially libraries) 437 from interfering with the new build. 438 To build to a DESTDIR with build.sh, use the "-D" option. 439 To build to a DESTDIR without using build.sh, set the DESTDIR 440 environment variable before running make build. It should be 441 set to the pathname of an initially empty directory. 442 Problems: if you do not use build.sh, you might need to 443 update critical utilities without using DESTDIR since 444 nothing is executed from what is installed in DESTDIR. 445 (See critical utils, below.) 446 Build often: 447 This keeps critical utilities current enough to not choke 448 on any other part of the source tree that depends on up to 449 date functionality. If you use build.sh, you should not have 450 this problem. 451 452 What to do if things don't work: 453 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 454 When things don't work there are usually a few things that commonly 455 should be done. 456 1) make includes 457 This should be done automatically by make build. 458 2) cd share/mk && make install 459 Again, automatically done by make build. 460 461 Failsafe rebuild of a small part of the tree: 462 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 463 To make sure you rebuild something correctly you want to do 464 something like the following: 465 1) Make sure the includes and .mk files are up to date. 466 2) Make sure any program used to build the particular 467 utility is up to date. (yacc, lex, etc...) 468 3) cd ...path/to/util... 469 make cleandir 470 rm ...all obj directories... 471 make cleandir # yes, again 472 make obj 473 make depend && make 474 475 Failsafe rebuild of the entire tree: 476 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 477 If you really want to make sure the source tree is clean and 478 ready for a build try the following. Note that sourcing /etc/mk.conf 479 (a make(1) Makefile) in this manner is not right, and will not work 480 for anyone who uses any make(1) features in /etc/mk.conf. 481 482 ---cut here--- 483 #!/bin/sh 484 . /etc/mk.conf 485 486 if [ -z $NETBSDSRCDIR ] ; then 487 NETBSDSRCDIR=/usr/src 488 fi 489 if [ \! -d $NETBSDSRCDIR ] ; then 490 echo Unable to find sources 491 exit 1 492 fi 493 find $NETBSDSRCDIR -name \*.o -o -name obj.\* -o -name obj -exec rm \{\} \; 494 495 if [ -z $BSDOBJDIR ] ; then 496 BSDOBJDIR=/usr/obj 497 fi 498 if [ -d $BSDOBJDIR ] ; then 499 rm -rf $BSDOBJDIR 500 fi 501 502 cd $NETBSDSRCDIR && make cleandir 503 504 ---cut here--- 505 506 Critical utilities: 507 ^^^^^^^^^^^^^^^^^^^ 508 usr.bin/make 509 usr.bin/yacc 510 usr.bin/lex 511 usr.bin/xlint 512 usr.bin/config 513 514 Other problems and possible solutions: 515 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 516 Symptom:Complaints involving a Makefile. 517 Fix: Rebuild usr.bin/make: 518 cd usr.bin/make && make && make install 519 Or, a failsafe method if that doesn't work: 520 cd usr.bin/make && cc -DMAKE_NATIVE *.c -I. -o make \ 521 && install make /usr/bin 522 523 Fix: Make sure .mk files are up to date. 524 cd share/mk && make install 525 526 Symptom:Kernel `config' fails to configure any kernel, including GENERIC. 527 Fix: Rebuild usr.bin/config 528 529 Symptom:Obsolete intermediate files are used during compilation 530 Fix: Try the following sequence of commands in the directory in question. 531 make cleandir; rm `make print-objdir`; make cleandir; make obj 532 (If you built the tree without "make obj" in the past, obsolete files 533 may remain. The command tries to clean everything up) 534 535 Symptom:.../sysinst/run.c:xx: warning: initialization from incompatible 536 pointer type 537 Fix: Rebuild and install usr.bin/menuc 538 539 Symptom:mklocale not found during build in share/locale/ctype 540 Fix: Build and install usr.bin/mklocale 541 542 Symptom:undefined reference to `__assert13' or `__unsetenv13' 543 Fix: Rebuild and install lib/libc 544 545 Symptom:usr.bin/config fails to build. 546 Fix: Try building with -DMAKE_BOOTSTRAP added to CFLAGS in Makefile. 547 548 Symptom:undefined reference to `getprogname' or `setprogname' 549 Fix: Rebuild and install lib/libc 550 551 Symptom:Update build fails in src/tools/gcc complaining that a variable 552 (e.g. CPPFLAGS) has changed since the previous run. 553 Fix: Run "nbmake-${ARCH} clean" in src/tools/gcc or do a clean build. 554 555 Symptom:cvs [update aborted]: cannot open directory /cvsroot/src/...: 556 No such file or directory. 557 Cause: If a directory is created by mistake, then it is sometimes 558 deleted from the CVS repository using administrative commands 559 that bypass the normal cvs access controls. If your cvs working tree 560 contains references to a directory that has been deleted on the 561 server in this way, then "cvs update" reports this error. 562 Fix: Recursively delete the affected directory from your working tree 563 and try the update again. 564
Indexes created Mon Oct 13 05:10:05 GMT 2025