UPDATING revision 1.365
1 $NetBSD: UPDATING,v 1.365 2024/11/10 13:58:59 martin Exp $ 2 3 This file (UPDATING) is intended to be a brief reference to recent 4 changes that might cause problems in the build process, and a guide for 5 what to do if something doesn't work. 6 7 For a more detailed description of the recommended way to build NetBSD 8 using build.sh, see the BUILDING file. 9 10 Note that much of the advice in this UPDATING file was written before 11 build.sh existed. Nevertheless, the advice here may be useful for 12 working around specific problems with build.sh. 13 14 Sections are marked with "^^^^^". After the section on "Recent changes" 15 are several sections containing more general information. 16 17 See also: BUILDING, build.sh, Makefile. 18 19 Recent changes: 20 ^^^^^^^^^^^^^^^ 21 22 20241104: 23 Changes around the integration of the zstd compression 24 library may require manual deletion of some object directories 25 if you update from a build from a few days ago. 26 rm -rf ${OBJ}/tools/gcc/ ${OBJ}/external/gpl3/gcc/ 27 rm -rf ${OBJ}/external/bsd/libarchive/ ${OBJ}/external/bsd/file/ 28 rm -rf ${OBJ}/external/bsd/zstd 29 Also remove libzstd from ${DEST}: rm ${DEST}/usr/lib/libzstd* 30 31 20240926: 32 Building -current when running on amd64 from about 20240924 33 to about 20240926 will not work due to a "malloc_np.h" 34 error. Please downgrade to a version from 20240924 or older, 35 or upgrade to a newer version e.g. using a binary release from 36 the NetBSD build servers. 37 38 20240923: 39 Update builds will require cleaning the object directories for 40 jemalloc (like: "rm -rf $OBJ/external/bsd/jemalloc*") and the 41 libc dependency files (like: "rm $OBJ/lib/libc/jemalloc.*"). 42 On architectures with COMPAT libs, those also have to be cleaned, 43 e.g. on sparc64: "rm -rf $OBJ/compat/sparc64/sparc/external/bsd/jemalloc*" 44 and "rm $OBJ/compat/sparc64/sparc/lib/libc/jemalloc.*". 45 46 20240817: 47 Update builds will require cleaning in src/external/historical/nawk/bin 48 in order to remove the awk binary since now there is a subdirectory 49 called awk. 50 51 20240812: 52 Update builds will require removing the tools objdir for gdb 53 if building with MKCROSSGDB=yes. All architectures use gdb.old 54 now. 55 56 20240630: 57 Update builds will require removing the tools objdir for binutils 58 (as most architectures should use binutils.old now). 59 60 20240610: 61 Update builds are likely to require removing the objdir for 62 distrib/<arch>/cdroms - or at least references to libarchive 63 from within there after the recent libarchive update. 64 65 20240504: 66 Update builds on x86 require making the `cleandir' target in 67 lib/libm, or deleting the objdir for lib/libm. 68 69 20240423: 70 Update builds with x11 may fail after switching to Mesa 21. 71 72 20240410: 73 Update builds may require manually deleting 74 $DESTDIR/var/run/named (and, potentially, 75 $DESTDIR/var/run/lwresd) in order to avoid checkflist failure. 76 77 20240409: 78 32-bit compatibility libraries were moved into the base32 79 and debug32 sets. HTML man pages were moved into the manhtml set. 80 81 20230828: 82 If: 83 - you updated to current and ran postinstall between 20230826 84 and 20230828 inclusive, and 85 - you are not using anything outside the base system to 86 populate /etc/openssl/certs (e.g., manually adding 87 certificates or using ca-certificates or mozilla-rootcerts or 88 mozilla-rootcerts-openssl from pkgsrc), 89 delete /etc/openssl/certs and rerun postinstall or just 90 `certctl rehash'. 91 92 Otherwise, certctl(8) will think someone else has left 93 /etc/openssl/certs in the way and will avoid clobbering it on 94 upgrade from older NetBSD. 95 96 20230718: 97 build.sh will use mk.conf in the same directory as build.sh instead 98 of /etc/mk.conf, unless MAKECONF is already defined in the environment. 99 100 20230714: 101 Import of gcc 10.5 requires a clean build of at least tools/gcc 102 and external/gpl3/gcc* object directories. An update build 103 without cleaning will result in obscure failures in rare corner 104 cases. 105 106 20230604: 107 Building native kernels with the installed make(1) instead of 108 the $TOOLDIR/bin/nbmake-${arch} wrapper will fail if userland 109 has not been updated. Note that this is an unsupported build 110 configuration (but usually works if userland is -current enough). 111 112 20230523: 113 ctfmerge was fixed as macOS host tool. This may require a clean tools 114 rebuild to detect the update to tools/compat/configure. 115 116 20230515: 117 New openssl build requires removal of the crypto/external/bsd/openssl 118 and tests/crypto/libcrypto subdirectories in the ${OBJDIR} hierarchy. 119 Otherwise test results may be wrong due to wrong "golden" output 120 files being installed. 121 122 20230322: 123 Fixes for moved /lib/libisns* mean that builds will complain about 124 ./usr/lib/libisns* being missing until src/lib/libisns is forced 125 to (re)install the files, including the symlinks. The simplest 126 way is to clean this subdir before building. 127 128 20230112: 129 New binutils require an updated ld.elf_so. If you are doing 130 (unsupported) in-place self builds (with the -E flag to build.sh), 131 make sure to have installed latest ld.elf_so before you rebuild 132 userland. 133 134 20221116: 135 The addition to NetBSD's version of UFS2 of support for extended 136 attributes broke backward compatibility with previous releases 137 of NetBSD, so UFS2 has been restored to being compatible with 138 previous NetBSD releases by disabling extended attributes. 139 (Note that ACLs are implemented as extended attributes, so 140 this changes disables ACLs as well.) 141 142 Support for UFS2 with extended attributes is now available in a new 143 UFS variant called UFS2ea. If you have created extended attributes 144 in an original UFS2 file system then "fsck -p" will now fail due to 145 the unexpected presence of extended attributes and "fsck -y" will 146 remove all extended attributes. If you wish to preserve extended 147 attributes rather than delete them, there is a utility to convert 148 a UFS2 file system to UFS2ea and leave extended attributes in place, 149 but this should be used with caution since it will preserve any 150 extended attributes that have been corrupted by the backward 151 incompatibility too. 152 153 If you wish to use a UFS2ea file system as your root file system, 154 then you will need to update your boot loader to a version that 155 supports UFS2ea. 156 157 For more information, see: 158 https://wiki.netbsd.org/features/UFS2ea 159 160 20221111: 161 The new libdrm import worsened the conflict issues for the 162 kdump/ktruss ioctl, and i915 now conflicts with base, and has 163 been turned off. This will cause update build issues like: 164 165 kdump-ioctl.c:12175:143: error: 'DRM_IOCTL_I915_DESTROY_HEAP' 166 undeclared here (not in a function); 167 did you mean 'DRM_IOCTL_MODE_DESTROY_DUMB'? 168 169 You'll need to clean usr.bin/ktruss, usr.bin/kdump, and rescue. 170 171 20220921: 172 Since the kernel version was bumped to 9.99.100, the bootloader 173 must be updated on x86 or EFI boot platforms in order to load 174 modules from boot.cfg(5) or interactively at the bootloader 175 prompt. 176 177 The kernel can still load modules at runtime, with modload(8) 178 or modules.conf(5), with no bootloader update. This will not 179 affect release branches because it only applies to patch 180 numbers >=100. 181 182 On x86 (i386 or amd64) with BIOS boot, this requires copying a 183 new /usr/mdec/boot to /boot. You can build this in the destdir 184 by running `$TOOLDIR/bin/nbmake-$ARCH dependall install' in 185 sys/arch/i386/stand. 186 187 On platforms with EFI boot (including x86 with EFI boot), this 188 requires copying a new /usr/mdec/boot*.efi to the EFI/BOOT/ 189 directory on the EFI system partition. The boot*.efi file is: 190 191 aarch64 bootaa64.efi 192 amd64 bootx64.efi 193 arm bootarm.efi (32-bit arm) 194 i386 bootx32.efi 195 196 You can build this in the destdir by running 197 `$TOOLDIR/bin/nbmake-$ARCH dependall install' in 198 sys/stand/efiboot/boot*. 199 200 20220821: 201 Support for building extsrc/ has been deprecated. 202 EXTSRCSRCDIR and MKEXTSRC have been deprecated. 203 204 20220714: 205 Updates to xorg-server and associated drivers may cause builds to 206 fail. Cleaning both DESTDIR and the xorg build tree may be needed. 207 208 20220628: 209 Changes in the build of src/games/robots require a clean build 210 of that program. 211 212 20211116: 213 Changes in the xorg build require clean obj dirs for external/mit/xorg 214 (or a full clean/non-update build). 215 216 20211112: 217 Device tree sources were updated for evbarm. Some device nodes 218 (in particular, ld(4) devices on the ROCKPro64) will be renumbered. 219 220 20211014: 221 MKPIE default for aarch64 has changed, a clean build is required. 222 223 20210917: 224 Changed MNT_ACLS to default to POSIX1e ACLs instead of NFSv4 225 ACLs, to be compatible with FreeBSD. If you are using NFSv4 ACLs 226 and have "acls" in /etc/fstab you'll need to change it to "nfs4acls". 227 228 20210711: 229 Updated GMP sources may require cleaning in tools/gmp and/or in 230 external/lgpl3/gmp/lib, particularly if GCC itself does not build. 231 232 20210417: 233 GCC 10 was enabled for several platforms. If builds fail in either 234 tools/gcc or external/gpl3/gcc, first try cleaning those objects and 235 removing the $DESTDIR/usr/include/g++ subdirectory. 236 237 20201016: 238 MIPS kernel modules have been disabled until they work. This will 239 turn up in extra files in the DESTDIR, which should be cleaned. 240 241 20200925: 242 GNU MPC and MPFR have been updated. At least MPFR needs cleaning 243 in both the tools and external dirs. 244 245 20200912: 246 GCC 9 has arrived for Arm and x86 platforms, and will be coming 247 for several more, as will binutils 2.34. Clean as required. 248 249 20200907: 250 GCC 9 is coming, and binutils has been updated for MIPS. This 251 probably requires cleaning the tools/binutils. 252 253 20200811: 254 GCC updates may require cleaning the objdir. This may occur 255 due to GCC 7.5 update, GCC 8.4 move to gcc.old, or the upcoming 256 GCC 9 upgrade. 257 258 20200614: 259 blacklist* has been renamed to blocklist*. postinstall(8) 260 should handle the migration. 261 262 20200601: 263 Due to a mistake in LIBISPRIVATE handling, .so libraries were 264 created in the build directories and need cleaning. 265 266 20200311: 267 GCC 8 ports will need cleaning in src/tools/gcc and 268 src/external/gpl3/gcc due to GCC 8.4 update. 269 270 20191118: 271 More architectures were switched to gcc8: 272 i386, ia64 powerpc64, sparc, sparc64, arm 273 The same comments as in 20191022 apply. 274 275 20191112: 276 The LLVM update requires a clean rebuild for all architectures using 277 LLVM during the tools build phase (i386, amd64, aarch64). 278 279 20191022: 280 Ports amd64 and aarch64 switched to GCC 8.3 by default. 281 In-place ("expert mode", build.sh -E) builds are not supported 282 when going from a GCC 7 userland to GCC 8. Do a regular 283 build to a different DESTDIR (or preferably: build.sh -U) at least 284 once and install sets, or download comp.{tar.xz,tgz} from the 285 daily builds and install that before doing the next in-place build. 286 287 20191001: 288 GCC 8.3 was imported. Builds of src/tools/gcc may fail if 289 old builds with GCC 7 output now uses GCC 8. Clean this 290 directory, and also clean src/external/gpl3/gcc. 291 292 20190903: 293 Files with names that coincide with existing files' names on 294 case-insensitive file systems were inadvertently committed, for 295 radeon GPU firmware. We cannot mark these as obsolete for 296 postinstall to fix, so if you updated src since 2019-08-26, and 297 ran build.sh distribution or ran build.sh release, you must 298 manually delete the following files in your DESTDIR (which is 299 usually $OBJDIR/destir.$ARCH), or from / if you have installed 300 them: 301 302 /libdata/firmware/radeon/bonaire_ce.bin 303 /libdata/firmware/radeon/bonaire_mc.bin 304 /libdata/firmware/radeon/bonaire_me.bin 305 /libdata/firmware/radeon/bonaire_mec.bin 306 /libdata/firmware/radeon/bonaire_pfp.bin 307 /libdata/firmware/radeon/bonaire_rlc.bin 308 /libdata/firmware/radeon/bonaire_sdma.bin 309 /libdata/firmware/radeon/bonaire_smc.bin 310 /libdata/firmware/radeon/bonaire_uvd.bin 311 /libdata/firmware/radeon/hainan_ce.bin 312 /libdata/firmware/radeon/hainan_mc.bin 313 /libdata/firmware/radeon/hainan_me.bin 314 /libdata/firmware/radeon/hainan_pfp.bin 315 /libdata/firmware/radeon/hainan_rlc.bin 316 /libdata/firmware/radeon/hainan_smc.bin 317 /libdata/firmware/radeon/hawaii_ce.bin 318 /libdata/firmware/radeon/hawaii_mc.bin 319 /libdata/firmware/radeon/hawaii_me.bin 320 /libdata/firmware/radeon/hawaii_mec.bin 321 /libdata/firmware/radeon/hawaii_pfp.bin 322 /libdata/firmware/radeon/hawaii_rlc.bin 323 /libdata/firmware/radeon/hawaii_sdma.bin 324 /libdata/firmware/radeon/hawaii_smc.bin 325 /libdata/firmware/radeon/kabini_ce.bin 326 /libdata/firmware/radeon/kabini_me.bin 327 /libdata/firmware/radeon/kabini_mec.bin 328 /libdata/firmware/radeon/kabini_pfp.bin 329 /libdata/firmware/radeon/kabini_rlc.bin 330 /libdata/firmware/radeon/kabini_sdma.bin 331 /libdata/firmware/radeon/kaveri_ce.bin 332 /libdata/firmware/radeon/kaveri_me.bin 333 /libdata/firmware/radeon/kaveri_mec.bin 334 /libdata/firmware/radeon/kaveri_mec2.bin 335 /libdata/firmware/radeon/kaveri_pfp.bin 336 /libdata/firmware/radeon/kaveri_rlc.bin 337 /libdata/firmware/radeon/kaveri_sdma.bin 338 /libdata/firmware/radeon/mullins_ce.bin 339 /libdata/firmware/radeon/mullins_me.bin 340 /libdata/firmware/radeon/mullins_mec.bin 341 /libdata/firmware/radeon/mullins_pfp.bin 342 /libdata/firmware/radeon/mullins_rlc.bin 343 /libdata/firmware/radeon/mullins_sdma.bin 344 /libdata/firmware/radeon/oland_ce.bin 345 /libdata/firmware/radeon/oland_mc.bin 346 /libdata/firmware/radeon/oland_me.bin 347 /libdata/firmware/radeon/oland_pfp.bin 348 /libdata/firmware/radeon/oland_rlc.bin 349 /libdata/firmware/radeon/oland_smc.bin 350 /libdata/firmware/radeon/pitcairn_ce.bin 351 /libdata/firmware/radeon/pitcairn_mc.bin 352 /libdata/firmware/radeon/pitcairn_me.bin 353 /libdata/firmware/radeon/pitcairn_pfp.bin 354 /libdata/firmware/radeon/pitcairn_rlc.bin 355 /libdata/firmware/radeon/pitcairn_smc.bin 356 /libdata/firmware/radeon/tahiti_ce.bin 357 /libdata/firmware/radeon/tahiti_mc.bin 358 /libdata/firmware/radeon/tahiti_me.bin 359 /libdata/firmware/radeon/tahiti_pfp.bin 360 /libdata/firmware/radeon/tahiti_rlc.bin 361 /libdata/firmware/radeon/tahiti_smc.bin 362 /libdata/firmware/radeon/verde_ce.bin 363 /libdata/firmware/radeon/verde_mc.bin 364 /libdata/firmware/radeon/verde_me.bin 365 /libdata/firmware/radeon/verde_pfp.bin 366 /libdata/firmware/radeon/verde_rlc.bin 367 /libdata/firmware/radeon/verde_smc.bin 368 369 We will re-import these radeon firmware images another way 370 later. 371 372 20190727: 373 The uefi bootloader has gained tftp support and needs a clean 374 build. If you do update builds, manually clean its object 375 directory by something like: 376 cd sys/arch/i386/stand/efiboot && make clean 377 378 20190723: 379 The jemalloc allocator in libc is now built without extended 380 debugging (for performance reasons). In update builds make sure 381 to rebuild it completely, by removing all affected object files, 382 including compat builds, something like: 383 cd /usr/obj && find . -type d -name jemalloc|xargs rm -rf 384 385 20190207: 386 GCC 7 switched for many ports. Update builds are likely to fail. 387 388 Hints for a more successful build: 389 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 390 Use build.sh, but do not use its "expert mode": 391 This will automatically build the tools in the 392 correct order, and it will keep the tools and the 393 new build products from interfering with the running 394 system. This will allow you to ignore most of the 395 other advice in this file. 396 Build a new kernel first: 397 This makes sure that any new system calls or features 398 expected by the new userland will be present. This 399 helps to avoid critical errors when upgrading. 400 Use object directories: 401 This helps to keep stale object 402 files from polluting the build if a Makefile "forgets" 403 about one. It also makes it easier to clean up after 404 a build. It's also necessary if you want to use the 405 same source tree for multiple machines. 406 To use object directories with build.sh: 407 a) invoke build.sh with the "-M" or "-O" options. 408 To use object directories without using build.sh: 409 a) cd /usr/src ; make cleandir 410 b) Add "OBJMACHINE=yes" to /etc/mk.conf 411 c) Add "MKOBJDIRS=yes" to /etc/mk.conf 412 d) cd /usr/src ; make build 413 Note that running "make obj" in a directory will create 414 in obj.$MACHINE directory. 415 Build to a DESTDIR: 416 This helps to keep old installed files (especially libraries) 417 from interfering with the new build. 418 To build to a DESTDIR with build.sh, use the "-D" option. 419 To build to a DESTDIR without using build.sh, set the DESTDIR 420 environment variable before running make build. It should be 421 set to the pathname of an initially empty directory. 422 Problems: if you do not use build.sh, you might need to 423 update critical utilities without using DESTDIR since 424 nothing is executed from what is installed in DESTDIR. 425 (See critical utils, below.) 426 Build often: 427 This keeps critical utilities current enough to not choke 428 on any other part of the source tree that depends on up to 429 date functionality. If you use build.sh, you should not have 430 this problem. 431 432 What to do if things don't work: 433 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 434 When things don't work there are usually a few things that commonly 435 should be done. 436 1) make includes 437 This should be done automatically by make build. 438 2) cd share/mk && make install 439 Again, automatically done by make build. 440 441 Failsafe rebuild of a small part of the tree: 442 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 443 To make sure you rebuild something correctly you want to do 444 something like the following: 445 1) Make sure the includes and .mk files are up to date. 446 2) Make sure any program used to build the particular 447 utility is up to date. (yacc, lex, etc...) 448 3) cd ...path/to/util... 449 make cleandir 450 rm ...all obj directories... 451 make cleandir # yes, again 452 make obj 453 make depend && make 454 455 Failsafe rebuild of the entire tree: 456 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 457 If you really want to make sure the source tree is clean and 458 ready for a build try the following. Note that sourcing /etc/mk.conf 459 (a make(1) Makefile) in this manner is not right, and will not work 460 for anyone who uses any make(1) features in /etc/mk.conf. 461 462 ---cut here--- 463 #!/bin/sh 464 . /etc/mk.conf 465 466 if [ -z $NETBSDSRCDIR ] ; then 467 NETBSDSRCDIR=/usr/src 468 fi 469 if [ \! -d $NETBSDSRCDIR ] ; then 470 echo Unable to find sources 471 exit 1 472 fi 473 find $NETBSDSRCDIR -name \*.o -o -name obj.\* -o -name obj -exec rm \{\} \; 474 475 if [ -z $BSDOBJDIR ] ; then 476 BSDOBJDIR=/usr/obj 477 fi 478 if [ -d $BSDOBJDIR ] ; then 479 rm -rf $BSDOBJDIR 480 fi 481 482 cd $NETBSDSRCDIR && make cleandir 483 484 ---cut here--- 485 486 Critical utilities: 487 ^^^^^^^^^^^^^^^^^^^ 488 usr.bin/make 489 usr.bin/yacc 490 usr.bin/lex 491 usr.bin/xlint 492 usr.bin/config 493 494 Other problems and possible solutions: 495 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 496 Symptom:Complaints involving a Makefile. 497 Fix: Rebuild usr.bin/make: 498 cd usr.bin/make && make && make install 499 Or, a failsafe method if that doesn't work: 500 cd usr.bin/make && cc -DMAKE_NATIVE *.c -I. -o make \ 501 && install make /usr/bin 502 503 Fix: Make sure .mk files are up to date. 504 cd share/mk && make install 505 506 Symptom:Kernel `config' fails to configure any kernel, including GENERIC. 507 Fix: Rebuild usr.bin/config 508 509 Symptom:Obsolete intermediate files are used during compilation 510 Fix: Try the following sequence of commands in the directory in question. 511 make cleandir; rm `make print-objdir`; make cleandir; make obj 512 (If you built the tree without "make obj" in the past, obsolete files 513 may remain. The command tries to clean everything up) 514 515 Symptom:.../sysinst/run.c:xx: warning: initialization from incompatible 516 pointer type 517 Fix: Rebuild and install usr.bin/menuc 518 519 Symptom:mklocale not found during build in share/locale/ctype 520 Fix: Build and install usr.bin/mklocale 521 522 Symptom:undefined reference to `__assert13' or `__unsetenv13' 523 Fix: Rebuild and install lib/libc 524 525 Symptom:usr.bin/config fails to build. 526 Fix: Try building with -DMAKE_BOOTSTRAP added to CFLAGS in Makefile. 527 528 Symptom:undefined reference to `getprogname' or `setprogname' 529 Fix: Rebuild and install lib/libc 530 531 Symptom:Update build fails in src/tools/gcc complaining that a variable 532 (e.g. CPPFLAGS) has changed since the previous run. 533 Fix: Run "nbmake-${ARCH} clean" in src/tools/gcc or do a clean build. 534 535 Symptom:cvs [update aborted]: cannot open directory /cvsroot/src/...: 536 No such file or directory. 537 Cause: If a directory is created by mistake, then it is sometimes 538 deleted from the CVS repository using administrative commands 539 that bypass the normal cvs access controls. If your cvs working tree 540 contains references to a directory that has been deleted on the 541 server in this way, then "cvs update" reports this error. 542 Fix: Recursively delete the affected directory from your working tree 543 and try the update again. 544
Indexes created Sun Oct 12 05:10:08 GMT 2025