UPDATING revision 1.366
1 $NetBSD: UPDATING,v 1.366 2024/11/11 13:58:56 riastradh Exp $ 2 3 This file (UPDATING) is intended to be a brief reference to recent 4 changes that might cause problems in the build process, and a guide for 5 what to do if something doesn't work. 6 7 For a more detailed description of the recommended way to build NetBSD 8 using build.sh, see the BUILDING file. 9 10 Note that much of the advice in this UPDATING file was written before 11 build.sh existed. Nevertheless, the advice here may be useful for 12 working around specific problems with build.sh. 13 14 Sections are marked with "^^^^^". After the section on "Recent changes" 15 are several sections containing more general information. 16 17 See also: BUILDING, build.sh, Makefile. 18 19 Recent changes: 20 ^^^^^^^^^^^^^^^ 21 22 20241104: 23 Changes around the integration of the zstd compression 24 library may require manual deletion of some object directories 25 if you update from a build from a few days ago: 26 rm -rf DESTDIR/usr/lib/*/libarchive* 27 rm -rf DESTDIR/usr/lib/*/libmagic* 28 rm -rf DESTDIR/usr/lib/*/libzstd* 29 rm -rf DESTDIR/usr/lib/libarchive* 30 rm -rf DESTDIR/usr/lib/libmagic* 31 rm -rf DESTDIR/usr/lib/libzstd* 32 rm -rf OBJDIR/compat/*/*/external/bsd/file/ 33 rm -rf OBJDIR/compat/*/*/external/bsd/libarchive/ 34 rm -rf OBJDIR/compat/*/*/external/bsd/zstd/ 35 rm -rf OBJDIR/external/bsd/file/ 36 rm -rf OBJDIR/external/bsd/libarchive/ 37 rm -rf OBJDIR/external/bsd/zstd/ 38 rm -rf OBJDIR/external/gpl3/gcc/ 39 rm -rf OBJDIR/tools/gcc/ 40 41 20240926: 42 Building -current when running on amd64 from about 20240924 43 to about 20240926 will not work due to a "malloc_np.h" 44 error. Please downgrade to a version from 20240924 or older, 45 or upgrade to a newer version e.g. using a binary release from 46 the NetBSD build servers. 47 48 20240923: 49 Update builds will require cleaning the object directories for 50 jemalloc (like: "rm -rf $OBJ/external/bsd/jemalloc*") and the 51 libc dependency files (like: "rm $OBJ/lib/libc/jemalloc.*"). 52 On architectures with COMPAT libs, those also have to be cleaned, 53 e.g. on sparc64: "rm -rf $OBJ/compat/sparc64/sparc/external/bsd/jemalloc*" 54 and "rm $OBJ/compat/sparc64/sparc/lib/libc/jemalloc.*". 55 56 20240817: 57 Update builds will require cleaning in src/external/historical/nawk/bin 58 in order to remove the awk binary since now there is a subdirectory 59 called awk. 60 61 20240812: 62 Update builds will require removing the tools objdir for gdb 63 if building with MKCROSSGDB=yes. All architectures use gdb.old 64 now. 65 66 20240630: 67 Update builds will require removing the tools objdir for binutils 68 (as most architectures should use binutils.old now). 69 70 20240610: 71 Update builds are likely to require removing the objdir for 72 distrib/<arch>/cdroms - or at least references to libarchive 73 from within there after the recent libarchive update. 74 75 20240504: 76 Update builds on x86 require making the `cleandir' target in 77 lib/libm, or deleting the objdir for lib/libm. 78 79 20240423: 80 Update builds with x11 may fail after switching to Mesa 21. 81 82 20240410: 83 Update builds may require manually deleting 84 $DESTDIR/var/run/named (and, potentially, 85 $DESTDIR/var/run/lwresd) in order to avoid checkflist failure. 86 87 20240409: 88 32-bit compatibility libraries were moved into the base32 89 and debug32 sets. HTML man pages were moved into the manhtml set. 90 91 20230828: 92 If: 93 - you updated to current and ran postinstall between 20230826 94 and 20230828 inclusive, and 95 - you are not using anything outside the base system to 96 populate /etc/openssl/certs (e.g., manually adding 97 certificates or using ca-certificates or mozilla-rootcerts or 98 mozilla-rootcerts-openssl from pkgsrc), 99 delete /etc/openssl/certs and rerun postinstall or just 100 `certctl rehash'. 101 102 Otherwise, certctl(8) will think someone else has left 103 /etc/openssl/certs in the way and will avoid clobbering it on 104 upgrade from older NetBSD. 105 106 20230718: 107 build.sh will use mk.conf in the same directory as build.sh instead 108 of /etc/mk.conf, unless MAKECONF is already defined in the environment. 109 110 20230714: 111 Import of gcc 10.5 requires a clean build of at least tools/gcc 112 and external/gpl3/gcc* object directories. An update build 113 without cleaning will result in obscure failures in rare corner 114 cases. 115 116 20230604: 117 Building native kernels with the installed make(1) instead of 118 the $TOOLDIR/bin/nbmake-${arch} wrapper will fail if userland 119 has not been updated. Note that this is an unsupported build 120 configuration (but usually works if userland is -current enough). 121 122 20230523: 123 ctfmerge was fixed as macOS host tool. This may require a clean tools 124 rebuild to detect the update to tools/compat/configure. 125 126 20230515: 127 New openssl build requires removal of the crypto/external/bsd/openssl 128 and tests/crypto/libcrypto subdirectories in the ${OBJDIR} hierarchy. 129 Otherwise test results may be wrong due to wrong "golden" output 130 files being installed. 131 132 20230322: 133 Fixes for moved /lib/libisns* mean that builds will complain about 134 ./usr/lib/libisns* being missing until src/lib/libisns is forced 135 to (re)install the files, including the symlinks. The simplest 136 way is to clean this subdir before building. 137 138 20230112: 139 New binutils require an updated ld.elf_so. If you are doing 140 (unsupported) in-place self builds (with the -E flag to build.sh), 141 make sure to have installed latest ld.elf_so before you rebuild 142 userland. 143 144 20221116: 145 The addition to NetBSD's version of UFS2 of support for extended 146 attributes broke backward compatibility with previous releases 147 of NetBSD, so UFS2 has been restored to being compatible with 148 previous NetBSD releases by disabling extended attributes. 149 (Note that ACLs are implemented as extended attributes, so 150 this changes disables ACLs as well.) 151 152 Support for UFS2 with extended attributes is now available in a new 153 UFS variant called UFS2ea. If you have created extended attributes 154 in an original UFS2 file system then "fsck -p" will now fail due to 155 the unexpected presence of extended attributes and "fsck -y" will 156 remove all extended attributes. If you wish to preserve extended 157 attributes rather than delete them, there is a utility to convert 158 a UFS2 file system to UFS2ea and leave extended attributes in place, 159 but this should be used with caution since it will preserve any 160 extended attributes that have been corrupted by the backward 161 incompatibility too. 162 163 If you wish to use a UFS2ea file system as your root file system, 164 then you will need to update your boot loader to a version that 165 supports UFS2ea. 166 167 For more information, see: 168 https://wiki.netbsd.org/features/UFS2ea 169 170 20221111: 171 The new libdrm import worsened the conflict issues for the 172 kdump/ktruss ioctl, and i915 now conflicts with base, and has 173 been turned off. This will cause update build issues like: 174 175 kdump-ioctl.c:12175:143: error: 'DRM_IOCTL_I915_DESTROY_HEAP' 176 undeclared here (not in a function); 177 did you mean 'DRM_IOCTL_MODE_DESTROY_DUMB'? 178 179 You'll need to clean usr.bin/ktruss, usr.bin/kdump, and rescue. 180 181 20220921: 182 Since the kernel version was bumped to 9.99.100, the bootloader 183 must be updated on x86 or EFI boot platforms in order to load 184 modules from boot.cfg(5) or interactively at the bootloader 185 prompt. 186 187 The kernel can still load modules at runtime, with modload(8) 188 or modules.conf(5), with no bootloader update. This will not 189 affect release branches because it only applies to patch 190 numbers >=100. 191 192 On x86 (i386 or amd64) with BIOS boot, this requires copying a 193 new /usr/mdec/boot to /boot. You can build this in the destdir 194 by running `$TOOLDIR/bin/nbmake-$ARCH dependall install' in 195 sys/arch/i386/stand. 196 197 On platforms with EFI boot (including x86 with EFI boot), this 198 requires copying a new /usr/mdec/boot*.efi to the EFI/BOOT/ 199 directory on the EFI system partition. The boot*.efi file is: 200 201 aarch64 bootaa64.efi 202 amd64 bootx64.efi 203 arm bootarm.efi (32-bit arm) 204 i386 bootx32.efi 205 206 You can build this in the destdir by running 207 `$TOOLDIR/bin/nbmake-$ARCH dependall install' in 208 sys/stand/efiboot/boot*. 209 210 20220821: 211 Support for building extsrc/ has been deprecated. 212 EXTSRCSRCDIR and MKEXTSRC have been deprecated. 213 214 20220714: 215 Updates to xorg-server and associated drivers may cause builds to 216 fail. Cleaning both DESTDIR and the xorg build tree may be needed. 217 218 20220628: 219 Changes in the build of src/games/robots require a clean build 220 of that program. 221 222 20211116: 223 Changes in the xorg build require clean obj dirs for external/mit/xorg 224 (or a full clean/non-update build). 225 226 20211112: 227 Device tree sources were updated for evbarm. Some device nodes 228 (in particular, ld(4) devices on the ROCKPro64) will be renumbered. 229 230 20211014: 231 MKPIE default for aarch64 has changed, a clean build is required. 232 233 20210917: 234 Changed MNT_ACLS to default to POSIX1e ACLs instead of NFSv4 235 ACLs, to be compatible with FreeBSD. If you are using NFSv4 ACLs 236 and have "acls" in /etc/fstab you'll need to change it to "nfs4acls". 237 238 20210711: 239 Updated GMP sources may require cleaning in tools/gmp and/or in 240 external/lgpl3/gmp/lib, particularly if GCC itself does not build. 241 242 20210417: 243 GCC 10 was enabled for several platforms. If builds fail in either 244 tools/gcc or external/gpl3/gcc, first try cleaning those objects and 245 removing the $DESTDIR/usr/include/g++ subdirectory. 246 247 20201016: 248 MIPS kernel modules have been disabled until they work. This will 249 turn up in extra files in the DESTDIR, which should be cleaned. 250 251 20200925: 252 GNU MPC and MPFR have been updated. At least MPFR needs cleaning 253 in both the tools and external dirs. 254 255 20200912: 256 GCC 9 has arrived for Arm and x86 platforms, and will be coming 257 for several more, as will binutils 2.34. Clean as required. 258 259 20200907: 260 GCC 9 is coming, and binutils has been updated for MIPS. This 261 probably requires cleaning the tools/binutils. 262 263 20200811: 264 GCC updates may require cleaning the objdir. This may occur 265 due to GCC 7.5 update, GCC 8.4 move to gcc.old, or the upcoming 266 GCC 9 upgrade. 267 268 20200614: 269 blacklist* has been renamed to blocklist*. postinstall(8) 270 should handle the migration. 271 272 20200601: 273 Due to a mistake in LIBISPRIVATE handling, .so libraries were 274 created in the build directories and need cleaning. 275 276 20200311: 277 GCC 8 ports will need cleaning in src/tools/gcc and 278 src/external/gpl3/gcc due to GCC 8.4 update. 279 280 20191118: 281 More architectures were switched to gcc8: 282 i386, ia64 powerpc64, sparc, sparc64, arm 283 The same comments as in 20191022 apply. 284 285 20191112: 286 The LLVM update requires a clean rebuild for all architectures using 287 LLVM during the tools build phase (i386, amd64, aarch64). 288 289 20191022: 290 Ports amd64 and aarch64 switched to GCC 8.3 by default. 291 In-place ("expert mode", build.sh -E) builds are not supported 292 when going from a GCC 7 userland to GCC 8. Do a regular 293 build to a different DESTDIR (or preferably: build.sh -U) at least 294 once and install sets, or download comp.{tar.xz,tgz} from the 295 daily builds and install that before doing the next in-place build. 296 297 20191001: 298 GCC 8.3 was imported. Builds of src/tools/gcc may fail if 299 old builds with GCC 7 output now uses GCC 8. Clean this 300 directory, and also clean src/external/gpl3/gcc. 301 302 20190903: 303 Files with names that coincide with existing files' names on 304 case-insensitive file systems were inadvertently committed, for 305 radeon GPU firmware. We cannot mark these as obsolete for 306 postinstall to fix, so if you updated src since 2019-08-26, and 307 ran build.sh distribution or ran build.sh release, you must 308 manually delete the following files in your DESTDIR (which is 309 usually $OBJDIR/destir.$ARCH), or from / if you have installed 310 them: 311 312 /libdata/firmware/radeon/bonaire_ce.bin 313 /libdata/firmware/radeon/bonaire_mc.bin 314 /libdata/firmware/radeon/bonaire_me.bin 315 /libdata/firmware/radeon/bonaire_mec.bin 316 /libdata/firmware/radeon/bonaire_pfp.bin 317 /libdata/firmware/radeon/bonaire_rlc.bin 318 /libdata/firmware/radeon/bonaire_sdma.bin 319 /libdata/firmware/radeon/bonaire_smc.bin 320 /libdata/firmware/radeon/bonaire_uvd.bin 321 /libdata/firmware/radeon/hainan_ce.bin 322 /libdata/firmware/radeon/hainan_mc.bin 323 /libdata/firmware/radeon/hainan_me.bin 324 /libdata/firmware/radeon/hainan_pfp.bin 325 /libdata/firmware/radeon/hainan_rlc.bin 326 /libdata/firmware/radeon/hainan_smc.bin 327 /libdata/firmware/radeon/hawaii_ce.bin 328 /libdata/firmware/radeon/hawaii_mc.bin 329 /libdata/firmware/radeon/hawaii_me.bin 330 /libdata/firmware/radeon/hawaii_mec.bin 331 /libdata/firmware/radeon/hawaii_pfp.bin 332 /libdata/firmware/radeon/hawaii_rlc.bin 333 /libdata/firmware/radeon/hawaii_sdma.bin 334 /libdata/firmware/radeon/hawaii_smc.bin 335 /libdata/firmware/radeon/kabini_ce.bin 336 /libdata/firmware/radeon/kabini_me.bin 337 /libdata/firmware/radeon/kabini_mec.bin 338 /libdata/firmware/radeon/kabini_pfp.bin 339 /libdata/firmware/radeon/kabini_rlc.bin 340 /libdata/firmware/radeon/kabini_sdma.bin 341 /libdata/firmware/radeon/kaveri_ce.bin 342 /libdata/firmware/radeon/kaveri_me.bin 343 /libdata/firmware/radeon/kaveri_mec.bin 344 /libdata/firmware/radeon/kaveri_mec2.bin 345 /libdata/firmware/radeon/kaveri_pfp.bin 346 /libdata/firmware/radeon/kaveri_rlc.bin 347 /libdata/firmware/radeon/kaveri_sdma.bin 348 /libdata/firmware/radeon/mullins_ce.bin 349 /libdata/firmware/radeon/mullins_me.bin 350 /libdata/firmware/radeon/mullins_mec.bin 351 /libdata/firmware/radeon/mullins_pfp.bin 352 /libdata/firmware/radeon/mullins_rlc.bin 353 /libdata/firmware/radeon/mullins_sdma.bin 354 /libdata/firmware/radeon/oland_ce.bin 355 /libdata/firmware/radeon/oland_mc.bin 356 /libdata/firmware/radeon/oland_me.bin 357 /libdata/firmware/radeon/oland_pfp.bin 358 /libdata/firmware/radeon/oland_rlc.bin 359 /libdata/firmware/radeon/oland_smc.bin 360 /libdata/firmware/radeon/pitcairn_ce.bin 361 /libdata/firmware/radeon/pitcairn_mc.bin 362 /libdata/firmware/radeon/pitcairn_me.bin 363 /libdata/firmware/radeon/pitcairn_pfp.bin 364 /libdata/firmware/radeon/pitcairn_rlc.bin 365 /libdata/firmware/radeon/pitcairn_smc.bin 366 /libdata/firmware/radeon/tahiti_ce.bin 367 /libdata/firmware/radeon/tahiti_mc.bin 368 /libdata/firmware/radeon/tahiti_me.bin 369 /libdata/firmware/radeon/tahiti_pfp.bin 370 /libdata/firmware/radeon/tahiti_rlc.bin 371 /libdata/firmware/radeon/tahiti_smc.bin 372 /libdata/firmware/radeon/verde_ce.bin 373 /libdata/firmware/radeon/verde_mc.bin 374 /libdata/firmware/radeon/verde_me.bin 375 /libdata/firmware/radeon/verde_pfp.bin 376 /libdata/firmware/radeon/verde_rlc.bin 377 /libdata/firmware/radeon/verde_smc.bin 378 379 We will re-import these radeon firmware images another way 380 later. 381 382 20190727: 383 The uefi bootloader has gained tftp support and needs a clean 384 build. If you do update builds, manually clean its object 385 directory by something like: 386 cd sys/arch/i386/stand/efiboot && make clean 387 388 20190723: 389 The jemalloc allocator in libc is now built without extended 390 debugging (for performance reasons). In update builds make sure 391 to rebuild it completely, by removing all affected object files, 392 including compat builds, something like: 393 cd /usr/obj && find . -type d -name jemalloc|xargs rm -rf 394 395 20190207: 396 GCC 7 switched for many ports. Update builds are likely to fail. 397 398 Hints for a more successful build: 399 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 400 Use build.sh, but do not use its "expert mode": 401 This will automatically build the tools in the 402 correct order, and it will keep the tools and the 403 new build products from interfering with the running 404 system. This will allow you to ignore most of the 405 other advice in this file. 406 Build a new kernel first: 407 This makes sure that any new system calls or features 408 expected by the new userland will be present. This 409 helps to avoid critical errors when upgrading. 410 Use object directories: 411 This helps to keep stale object 412 files from polluting the build if a Makefile "forgets" 413 about one. It also makes it easier to clean up after 414 a build. It's also necessary if you want to use the 415 same source tree for multiple machines. 416 To use object directories with build.sh: 417 a) invoke build.sh with the "-M" or "-O" options. 418 To use object directories without using build.sh: 419 a) cd /usr/src ; make cleandir 420 b) Add "OBJMACHINE=yes" to /etc/mk.conf 421 c) Add "MKOBJDIRS=yes" to /etc/mk.conf 422 d) cd /usr/src ; make build 423 Note that running "make obj" in a directory will create 424 in obj.$MACHINE directory. 425 Build to a DESTDIR: 426 This helps to keep old installed files (especially libraries) 427 from interfering with the new build. 428 To build to a DESTDIR with build.sh, use the "-D" option. 429 To build to a DESTDIR without using build.sh, set the DESTDIR 430 environment variable before running make build. It should be 431 set to the pathname of an initially empty directory. 432 Problems: if you do not use build.sh, you might need to 433 update critical utilities without using DESTDIR since 434 nothing is executed from what is installed in DESTDIR. 435 (See critical utils, below.) 436 Build often: 437 This keeps critical utilities current enough to not choke 438 on any other part of the source tree that depends on up to 439 date functionality. If you use build.sh, you should not have 440 this problem. 441 442 What to do if things don't work: 443 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 444 When things don't work there are usually a few things that commonly 445 should be done. 446 1) make includes 447 This should be done automatically by make build. 448 2) cd share/mk && make install 449 Again, automatically done by make build. 450 451 Failsafe rebuild of a small part of the tree: 452 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 453 To make sure you rebuild something correctly you want to do 454 something like the following: 455 1) Make sure the includes and .mk files are up to date. 456 2) Make sure any program used to build the particular 457 utility is up to date. (yacc, lex, etc...) 458 3) cd ...path/to/util... 459 make cleandir 460 rm ...all obj directories... 461 make cleandir # yes, again 462 make obj 463 make depend && make 464 465 Failsafe rebuild of the entire tree: 466 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 467 If you really want to make sure the source tree is clean and 468 ready for a build try the following. Note that sourcing /etc/mk.conf 469 (a make(1) Makefile) in this manner is not right, and will not work 470 for anyone who uses any make(1) features in /etc/mk.conf. 471 472 ---cut here--- 473 #!/bin/sh 474 . /etc/mk.conf 475 476 if [ -z $NETBSDSRCDIR ] ; then 477 NETBSDSRCDIR=/usr/src 478 fi 479 if [ \! -d $NETBSDSRCDIR ] ; then 480 echo Unable to find sources 481 exit 1 482 fi 483 find $NETBSDSRCDIR -name \*.o -o -name obj.\* -o -name obj -exec rm \{\} \; 484 485 if [ -z $BSDOBJDIR ] ; then 486 BSDOBJDIR=/usr/obj 487 fi 488 if [ -d $BSDOBJDIR ] ; then 489 rm -rf $BSDOBJDIR 490 fi 491 492 cd $NETBSDSRCDIR && make cleandir 493 494 ---cut here--- 495 496 Critical utilities: 497 ^^^^^^^^^^^^^^^^^^^ 498 usr.bin/make 499 usr.bin/yacc 500 usr.bin/lex 501 usr.bin/xlint 502 usr.bin/config 503 504 Other problems and possible solutions: 505 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 506 Symptom:Complaints involving a Makefile. 507 Fix: Rebuild usr.bin/make: 508 cd usr.bin/make && make && make install 509 Or, a failsafe method if that doesn't work: 510 cd usr.bin/make && cc -DMAKE_NATIVE *.c -I. -o make \ 511 && install make /usr/bin 512 513 Fix: Make sure .mk files are up to date. 514 cd share/mk && make install 515 516 Symptom:Kernel `config' fails to configure any kernel, including GENERIC. 517 Fix: Rebuild usr.bin/config 518 519 Symptom:Obsolete intermediate files are used during compilation 520 Fix: Try the following sequence of commands in the directory in question. 521 make cleandir; rm `make print-objdir`; make cleandir; make obj 522 (If you built the tree without "make obj" in the past, obsolete files 523 may remain. The command tries to clean everything up) 524 525 Symptom:.../sysinst/run.c:xx: warning: initialization from incompatible 526 pointer type 527 Fix: Rebuild and install usr.bin/menuc 528 529 Symptom:mklocale not found during build in share/locale/ctype 530 Fix: Build and install usr.bin/mklocale 531 532 Symptom:undefined reference to `__assert13' or `__unsetenv13' 533 Fix: Rebuild and install lib/libc 534 535 Symptom:usr.bin/config fails to build. 536 Fix: Try building with -DMAKE_BOOTSTRAP added to CFLAGS in Makefile. 537 538 Symptom:undefined reference to `getprogname' or `setprogname' 539 Fix: Rebuild and install lib/libc 540 541 Symptom:Update build fails in src/tools/gcc complaining that a variable 542 (e.g. CPPFLAGS) has changed since the previous run. 543 Fix: Run "nbmake-${ARCH} clean" in src/tools/gcc or do a clean build. 544 545 Symptom:cvs [update aborted]: cannot open directory /cvsroot/src/...: 546 No such file or directory. 547 Cause: If a directory is created by mistake, then it is sometimes 548 deleted from the CVS repository using administrative commands 549 that bypass the normal cvs access controls. If your cvs working tree 550 contains references to a directory that has been deleted on the 551 server in this way, then "cvs update" reports this error. 552 Fix: Recursively delete the affected directory from your working tree 553 and try the update again. 554
Indexes created Mon Oct 13 16:09:52 GMT 2025