BUILDING revision 1.156
11.104SwizBUILDING(8)                 System Manager's Manual                BUILDING(8)
21.1Stv
31.105SwizNAME
41.150Slukem     BUILDING - Procedure for building NetBSD from source code
51.1Stv
61.105SwizREQUIREMENTS
71.1Stv     NetBSD is designed to be buildable on most POSIX-compliant host systems.
81.105Swiz     The basic build procedure is the same whether compiling natively (on the
91.105Swiz     same NetBSD architecture) or cross compiling (on another architecture or
101.1Stv     OS).
111.1Stv
121.126Swiz     This source tree contains a special subtree, "tools", which uses the host
131.126Swiz     system to create a build toolchain for the target architecture.  The host
141.126Swiz     system must have at least C and C++ compilers in order to create the
151.105Swiz     toolchain (make is not required); all other tools are created as part of
161.147Slukem     the NetBSD build process.  (See the Environment variables section below
171.53Sreed     if you need to override or manually select your compilers.)
181.1Stv
191.155Slukem     Note: Within this document, cross-references to manual pages are to the
201.155Slukem     NetBSD manual pages, not the host system manual pages.  The mdoc(7)
211.155Slukem     source to the NetBSD manual pages can be found within the source tree,
221.155Slukem     and these and can be formatted with mandoc(1) or nroff(1) if those are
231.155Slukem     available on the host system.  The NetBSD manual pages are also available
241.155Slukem     at https://man.netbsd.org
251.155Slukem
261.105SwizFILES
271.105Swiz   Source tree layout
281.151Slukem     BUILDING       This document (in plaintext).  Generated from
291.151Slukem                    doc/BUILDING.mdoc.
301.48Sjmc
311.105Swiz     Makefile       The main Makefile for NetBSD; should only be run for
321.44Slukem                    native builds with an appropriately up-to-date version of
331.142Sandvar                    NetBSD make(1).  Intended for expert use with knowledge of
341.129Ssevan                    its shortcomings, it has been superseded by the build.sh
351.129Ssevan                    shell script as the recommended means for building NetBSD.
361.1Stv
371.105Swiz     UPDATING       Special notes for updating from an earlier revision of
381.1Stv                    NetBSD.  It is important to read this file before every
391.1Stv                    build of an updated source tree.
401.1Stv
411.105Swiz     build.sh       Bourne-compatible shell script used for building the host
421.1Stv                    build tools and the NetBSD system from scratch.  Can be
431.1Stv                    used for both native and cross builds, and should be used
441.129Ssevan                    instead of make(1) as it performs additional checks to
451.129Ssevan                    prevent common issues going undetected, such as building
461.129Ssevan                    with an outdated version of make(1).
471.1Stv
481.105Swiz     crypto/dist/, dist/, gnu/dist/
491.104Swiz                    Sources imported verbatim from third parties, without
501.104Swiz                    mangling the existing build structure.  Other source trees
511.126Swiz                    in bin through usr.sbin use the NetBSD make(1) "reachover"
521.126Swiz                    Makefile semantics when building these programs for a
531.126Swiz                    native host.
541.104Swiz
551.151Slukem     distrib/, etc/
561.151Slukem                    Sources for items used when making a full release
571.151Slukem                    snapshot, such as files installed in DESTDIR/etc on the
581.151Slukem                    destination system, boot media, and release notes.
591.151Slukem
601.151Slukem     doc/BUILDING.mdoc
611.155Slukem                    The source to this document, in mdoc(7) format.  Used to
621.155Slukem                    generate BUILDING.
631.151Slukem
641.106Swiz     external, sys/external
651.106Swiz                    Sources and build infrastructure for components imported
661.106Swiz                    (mostly) unchanged from upstream maintainers, sorted by
671.106Swiz                    applicable license.  This is (slowly) replacing the
681.106Swiz                    crypto/dist, dist, and gnu/dist directories.
691.106Swiz
701.151Slukem     external/mit/xorg/
711.151Slukem                    "Reachover" build structure for modular Xorg; the source
721.151Slukem                    is in X11SRCDIR.
731.1Stv
741.151Slukem     regress/, tests/
751.70Sdholland                    Regression test harness.  Can be cross-compiled, but only
761.105Swiz                    run natively.  tests/ uses the atf(7) test framework;
771.105Swiz                    regress/ contains older tests that have not yet been
781.70Sdholland                    migrated to atf(7).
791.1Stv
801.105Swiz     sys/           NetBSD kernel sources.
811.1Stv
821.126Swiz     tools/         "Reachover" build structure for the host build tools.
831.104Swiz                    This has a special method of determining out-of-date
841.104Swiz                    status.
851.1Stv
861.151Slukem     tools/compat/README
871.151Slukem                    Special notes for cross-hosting a NetBSD build on non-
881.151Slukem                    NetBSD platforms.
891.151Slukem
901.151Slukem     Other directories including bin/ ... usr.sbin/
911.1Stv                    Sources to the NetBSD userland (non-kernel) programs.  If
921.1Stv                    any of these directories are missing, they will be skipped
931.1Stv                    during the build.
941.1Stv
951.105Swiz   Build tree layout
961.156Slukem     The NetBSD build tree is described in hier(7) (whose mdoc(7) source is in
971.155Slukem     share/man/man7/hier.7), and the release layout is described in release(7)
981.156Slukem     (whose mdoc(7) source is in share/man/man7/release.7).
991.1Stv
1001.105SwizCONFIGURATION
1011.105Swiz   Environment variables
1021.8Slukem     Several environment variables control the behaviour of NetBSD builds.
1031.8Slukem
1041.152Slukem     HOST_CC         Path name to C compiler used to create the toolchain.
1051.58Sapb
1061.155Slukem                     Default: "cc".
1071.155Slukem
1081.152Slukem     HOST_CFLAGS     Flags passed to the host C compiler.
1091.53Sreed
1101.155Slukem                     Default: "-O".
1111.155Slukem
1121.155Slukem     HOST_CPPFLAGS   Flags passed to the host C/C++ pre-processor.
1131.155Slukem
1141.155Slukem                     Default: Unset.
1151.155Slukem
1161.152Slukem     HOST_CXX        Path name to C++ compiler used to create the toolchain.
1171.137Suwe
1181.155Slukem                     Default: Unset, but defaults to "c++" where required.
1191.155Slukem
1201.152Slukem     HOST_CXXFLAGS   Flags passed to the host C++ compiler.
1211.53Sreed
1221.155Slukem                     Default: Unset.
1231.155Slukem
1241.152Slukem     HOST_SH         Path name to a shell available on the host system and
1251.152Slukem                     suitable for use during the build.  The NetBSD build
1261.152Slukem                     system requires a modern Bourne-like shell with POSIX-
1271.152Slukem                     compliant features, and also requires support for the
1281.152Slukem                     "local" keyword to declare local variables in shell
1291.152Slukem                     functions (which is a widely-implemented but non-
1301.152Slukem                     standardised feature).
1311.152Slukem
1321.152Slukem                     Depending on the host system, a suitable shell may be
1331.152Slukem                     /bin/sh, /usr/xpg4/bin/sh, /bin/ksh (provided it is a
1341.152Slukem                     variant of ksh that supports the "local" keyword, such as
1351.152Slukem                     ksh88, but not ksh93), or /usr/local/bin/bash.
1361.152Slukem
1371.152Slukem                     Most parts of the build require HOST_SH to be an absolute
1381.152Slukem                     path; however, build.sh allows it to be a simple command
1391.152Slukem                     name, which will be converted to an absolute path by
1401.152Slukem                     searching the PATH.
1411.137Suwe
1421.155Slukem                     Default: "sh".
1431.155Slukem
1441.147Slukem     INSTALLBOOT_UBOOT_PATHS
1451.152Slukem                     A colon-separated list of search paths used by
1461.152Slukem                     installboot(8) to find U-Boot packages.
1471.147Slukem
1481.155Slukem                     Default: Unset.
1491.155Slukem
1501.152Slukem     MACHINE         Machine type, e.g., "macppc".
1511.8Slukem
1521.155Slukem                     Default: Unset.
1531.155Slukem
1541.152Slukem     MACHINE_ARCH    Machine architecture, e.g., "powerpc".
1551.8Slukem
1561.155Slukem                     Default: Unset.
1571.155Slukem
1581.152Slukem     MAKE            Path name to invoke make(1) as.
1591.8Slukem
1601.155Slukem                     Default: "make".
1611.155Slukem
1621.152Slukem     MAKECONF        The name of the make(1) configuration file.  See "make"
1631.152Slukem                     variables and mk.conf(5).
1641.151Slukem
1651.152Slukem                     Note: Only settable in the process environment.
1661.151Slukem
1671.155Slukem                     Default: "/etc/mk.conf".
1681.151Slukem
1691.152Slukem     MAKEFLAGS       Flags to invoke make(1) with.
1701.151Slukem
1711.152Slukem                     Note: build.sh ignores the value of MAKEFLAGS passed in
1721.152Slukem                     the environment, but allows MAKEFLAGS to be set via the
1731.152Slukem                     -V option.
1741.151Slukem
1751.155Slukem                     Default: "-X" on systems with a small ARG_MAX (Cygwin,
1761.155Slukem                     Darwin, FreeBSD); otherwise unset.
1771.155Slukem
1781.152Slukem     MAKEOBJDIR      Directory to use as the .OBJDIR for the current
1791.152Slukem                     directory.  The value is subjected to variable expansion
1801.152Slukem                     by make(1).  Typical usage is to set this variable to a
1811.152Slukem                     value involving the use of `${.CURDIR:S...}' or
1821.152Slukem                     `${.CURDIR:C...}', to derive the value of .OBJDIR from
1831.152Slukem                     the value of .CURDIR.  Used only if MAKEOBJDIRPREFIX is
1841.152Slukem                     not defined.
1851.151Slukem
1861.152Slukem                     Note: MAKEOBJDIR can be provided only in the environment
1871.152Slukem                     or via the -O flag of build.sh; it cannot usefully be set
1881.152Slukem                     inside a Makefile, including in mk.conf(5) or MAKECONF.
1891.151Slukem
1901.155Slukem                     Default: Unset.
1911.155Slukem
1921.151Slukem     MAKEOBJDIRPREFIX
1931.152Slukem                     Top level directory of the object directory tree.  The
1941.152Slukem                     value is subjected to variable expansion by make(1).
1951.152Slukem                     build.sh will create the ${MAKEOBJDIRPREFIX} directory if
1961.152Slukem                     necessary, but if make(1) is used without build.sh, then
1971.152Slukem                     rules in <bsd.obj.mk> will abort the build if the
1981.152Slukem                     ${MAKEOBJDIRPREFIX} directory does not exist.  If the
1991.152Slukem                     value is defined and valid, then
2001.152Slukem                     ${MAKEOBJDIRPREFIX}/${.CURDIR} is used as the .OBJDIR for
2011.152Slukem                     the current directory.  The current directory may be read
2021.152Slukem                     only.
2031.152Slukem
2041.152Slukem                     Note: MAKEOBJDIRPREFIX can be provided only in the
2051.152Slukem                     environment or via the -M flag of build.sh; it cannot
2061.152Slukem                     usefully be set inside a Makefile, including in
2071.152Slukem                     mk.conf(5) or MAKECONF.
2081.152Slukem
2091.155Slukem                     Default: Unset.
2101.155Slukem
2111.152Slukem     TMPDIR          Top-level directory to store temporary directories used
2121.152Slukem                     by build.sh before paths to other directories such as
2131.152Slukem                     .OBJDIR can be determined.
2141.149Slukem
2151.152Slukem                     Note: Must support execution of binaries.  I.e., without
2161.152Slukem                     mount(8)'s -o noexec option.
2171.149Slukem
2181.152Slukem                     Default: "/tmp".
2191.149Slukem
2201.105Swiz   "make" variables
2211.156Slukem     Variables that control the behavior of NetBSD builds are documented in
2221.156Slukem     mk.conf(5) (whose mdoc(7) source is in share/man/man5/mk.conf.5).
2231.45Slukem
2241.155Slukem     Unless otherwise specified, these variables may be set in either the
2251.155Slukem     process environment or the make(1) configuration file mk.conf(5)
2261.155Slukem     specified by MAKECONF.
2271.49Slukem
2281.105SwizBUILDING
2291.105Swiz   "make" command line options
2301.52Sjmc     This is not a summary of all the options available to make(1); only the
2311.52Sjmc     options used most frequently with NetBSD builds are listed here.
2321.1Stv
2331.105Swiz     -j njob    Run up to njob make(1) subjobs in parallel.  Makefiles should
2341.52Sjmc                use .WAIT or have explicit dependencies as necessary to
2351.74Sapb                enforce build ordering.
2361.10Ssommerfe
2371.105Swiz     -m dir     Specify the default directory for searching for system
2381.104Swiz                Makefile segments, mainly the <bsd.*.mk> files.  When building
2391.104Swiz                any full NetBSD source tree, this should be set to the
2401.126Swiz                "share/mk" directory in the source tree.  This is set
2411.104Swiz                automatically when building from the top level, or when using
2421.105Swiz                build.sh.
2431.1Stv
2441.148Slukem     -n         Show the commands that would have been executed, but do not
2451.1Stv                actually execute them.  This will still cause recursion to
2461.1Stv                take place.
2471.1Stv
2481.148Slukem     -V var     Show make(1)'s idea of the value of var.  Does not build any
2491.1Stv                targets.
2501.1Stv
2511.105Swiz     var=value  Set the variable var to value, overriding any setting
2521.105Swiz                specified by the process environment, the MAKECONF
2531.104Swiz                configuration file, or the system Makefile segments.
2541.1Stv
2551.105Swiz   "make" targets
2561.1Stv     These default targets may be built by running make(1) in any subtree of
2571.1Stv     the NetBSD source code.  It is recommended that none of these be used
2581.126Swiz     from the top level Makefile; as a specific exception, "make obj" and
2591.126Swiz     "make cleandir" are useful in that context.
2601.1Stv
2611.105Swiz     all        Build programs, libraries, and preformatted documentation.
2621.1Stv
2631.105Swiz     clean      Remove program and library object code files.
2641.1Stv
2651.105Swiz     cleandir   Same as clean, but also remove preformatted documentation,
2661.126Swiz                dependency files generated by "make depend", and any other
2671.49Slukem                files known to be created at build time.
2681.1Stv
2691.105Swiz     depend     Create dependency files (.depend) containing more detailed
2701.44Slukem                information about the dependencies of source code on header
2711.1Stv                files.  Allows programs to be recompiled automatically when a
2721.1Stv                dependency changes.
2731.1Stv
2741.126Swiz     dependall  Does a "make depend" immediately followed by a "make all".
2751.10Ssommerfe                This improves cache locality of the build since both passes
2761.10Ssommerfe                read the source files in their entirety.
2771.1Stv
2781.105Swiz     distclean  Synonym for cleandir.
2791.49Slukem
2801.105Swiz     includes   Build and install system header files.  Typically needed
2811.44Slukem                before any system libraries or programs can be built.
2821.1Stv
2831.105Swiz     install    Install programs, libraries, and documentation into DESTDIR.
2841.105Swiz                Few files will be installed to DESTDIR/dev, DESTDIR/etc,
2851.105Swiz                DESTDIR/root or DESTDIR/var in order to prevent user supplied
2861.27Slukem                configuration data from being overwritten.
2871.1Stv
2881.105Swiz     lint       Run lint(1) against the C source code, where appropriate, and
2891.1Stv                generate system-installed lint libraries.
2901.1Stv
2911.105Swiz     obj        Create object directories to be used for built files, instead
2921.1Stv                of building directly in the source tree.
2931.1Stv
2941.105Swiz     tags       Create ctags(1) searchable function lists usable by the ex(1)
2951.1Stv                and vi(1) text editors.
2961.1Stv
2971.105Swiz   "make" targets for the top level
2981.1Stv     Additional make(1) targets are usable specifically from the top source
2991.1Stv     level to facilitate building the entire NetBSD source tree.
3001.1Stv
3011.105Swiz     build         Build the entire NetBSD system (except the kernel).  This
3021.68Sreed                   orders portions of the source tree such that prerequisites
3031.68Sreed                   will be built in the proper order.
3041.20Slukem
3051.126Swiz     distribution  Do a "make build", and then install a full distribution
3061.105Swiz                   (which does not include a kernel) into DESTDIR, including
3071.105Swiz                   files in DESTDIR/dev, DESTDIR/etc, DESTDIR/root and
3081.105Swiz                   DESTDIR/var.
3091.20Slukem
3101.126Swiz     buildworld    As per "make distribution", except that it ensures that
3111.105Swiz                   DESTDIR is not the root directory.
3121.20Slukem
3131.105Swiz     installworld  Install the distribution from DESTDIR to INSTALLWORLDDIR,
3141.70Sdholland                   which defaults to the root directory.  Ensures that
3151.105Swiz                   INSTALLWORLDDIR is not the root directory if cross
3161.104Swiz                   compiling.
3171.20Slukem
3181.105Swiz                   The INSTALLSETS environment variable may be set to a space-
3191.95Sapb                   separated list of distribution sets to be installed.  By
3201.126Swiz                   default, all sets except "etc" and "xetc" are installed, so
3211.126Swiz                   most files in INSTALLWORLDDIR/etc will not be installed or
3221.126Swiz                   modified.
3231.61Sapb
3241.105Swiz                   Note: Before performing this operation with
3251.105Swiz                   INSTALLWORLDDIR=/, it is highly recommended that you
3261.61Sapb                   upgrade your kernel and reboot.  After performing this
3271.61Sapb                   operation, it is recommended that you use etcupdate(8) to
3281.105Swiz                   update files in INSTALLWORLDDIR/etc, and postinstall(8) to
3291.102Sabs                   check for or fix inconsistencies.
3301.20Slukem
3311.105Swiz     sets          Create distribution sets from DESTDIR into
3321.105Swiz                   RELEASEDIR/RELEASEMACHINEDIR/binary/sets.  Should be run
3331.126Swiz                   after "make distribution", as "make build" alone does not
3341.126Swiz                   install all of the required files.
3351.25Slukem
3361.105Swiz     sourcesets    Create source sets of the source tree into
3371.105Swiz                   RELEASEDIR/source/sets.
3381.31Slukem
3391.105Swiz     syspkgs       Create syspkgs from DESTDIR into
3401.105Swiz                   RELEASEDIR/RELEASEMACHINEDIR/binary/syspkgs.  Should be run
3411.126Swiz                   after "make distribution", as "make build" alone does not
3421.126Swiz                   install all of the required files.
3431.54Sapb
3441.126Swiz     release       Do a "make distribution", build kernels, distribution
3451.126Swiz                   media, and install sets (this as per "make sets"), and then
3461.126Swiz                   package the system into a standard release layout as
3471.105Swiz                   described by release(7).  This requires that RELEASEDIR be
3481.25Slukem                   set (see above).
3491.1Stv
3501.105Swiz     iso-image     Create a NetBSD installation CD-ROM image in the
3511.114Ssnj                   RELEASEDIR/images directory.  The CD-ROM file system will
3521.114Ssnj                   have a layout as described in release(7).
3531.62Sapb
3541.62Sapb                   For most machine types, the CD-ROM will be bootable, and
3551.104Swiz                   will automatically run the sysinst(8) menu-based
3561.104Swiz                   installation program, which can be used to install or
3571.104Swiz                   upgrade a NetBSD system.  Bootable CD-ROMs also contain
3581.104Swiz                   tools that may be useful in repairing a damaged NetBSD
3591.104Swiz                   installation.
3601.62Sapb
3611.126Swiz                   Before "make iso-image" is attempted, RELEASEDIR must be
3621.126Swiz                   populated by "make release" or equivalent.
3631.62Sapb
3641.151Slukem                   Note: Other, smaller, CD-ROM images may be created in the
3651.151Slukem                   RELEASEDIR/RELEASEMACHINEDIR/installation/cdrom directory
3661.151Slukem                   by "make release".  These smaller images usually contain
3671.151Slukem                   the same tools as the larger images in RELEASEDIR/images,
3681.151Slukem                   but do not contain additional content such as the
3691.151Slukem                   distribution sets.
3701.62Sapb
3711.151Slukem                   Note: The mac68k port still uses an older method of
3721.104Swiz                   creating CD-ROM images.  This requires the mkisofs(1)
3731.104Swiz                   utility, which is not part of NetBSD, but which can be
3741.105Swiz                   installed from pkgsrc/sysutils/cdrtools.
3751.56Sapb
3761.105Swiz     iso-image-source
3771.64Sjnemeth                   Create a NetBSD installation CD-ROM image in the
3781.114Ssnj                   RELEASEDIR/images directory.  The CD-ROM file system will
3791.114Ssnj                   have a layout as described in release(7).  It will have top
3801.64Sjnemeth                   level directories for the machine type and source.
3811.64Sjnemeth
3821.64Sjnemeth                   For most machine types, the CD-ROM will be bootable, and
3831.104Swiz                   will automatically run the sysinst(8) menu-based
3841.104Swiz                   installation program, which can be used to install or
3851.104Swiz                   upgrade a NetBSD system.  Bootable CD-ROMs also contain
3861.104Swiz                   tools that may be useful in repairing a damaged NetBSD
3871.104Swiz                   installation.
3881.64Sjnemeth
3891.126Swiz                   Before "make iso-image-source" is attempted, RELEASEDIR
3901.126Swiz                   must be populated by "make sourcesets release" or
3911.104Swiz                   equivalent.
3921.64Sjnemeth
3931.151Slukem                   Note: Other, smaller, CD-ROM images may be created in the
3941.151Slukem                   RELEASEDIR/RELEASEMACHINEDIR/installation/cdrom directory
3951.151Slukem                   by "make release".  These smaller images usually contain
3961.151Slukem                   the same tools as the larger images in RELEASEDIR/images,
3971.151Slukem                   but do not contain additional content such as the
3981.151Slukem                   distribution sets.
3991.64Sjnemeth
4001.151Slukem                   Note: The mac68k port still uses an older method of
4011.104Swiz                   creating CD-ROM images.  This requires the mkisofs(1)
4021.104Swiz                   utility, which is not part of NetBSD, but which can be
4031.105Swiz                   installed from pkgsrc/sysutils/cdrtools.
4041.64Sjnemeth
4051.105Swiz     install-image
4061.98Sapb                   Create a bootable NetBSD installation disk image in the
4071.127Splunky                   RELEASEDIR/images directory.  The installation disk image
4081.127Splunky                   is suitable for copying to bootable USB flash memory
4091.127Splunky                   sticks, etc., for machines which are able to boot from such
4101.127Splunky                   devices.  The file system in the bootable disk image will
4111.127Splunky                   have a layout as described in release(7).
4121.97Stsutsui
4131.97Stsutsui                   The installation image is bootable, and will automatically
4141.97Stsutsui                   run the sysinst(8) menu-based installation program, which
4151.98Sapb                   can be used to install or upgrade a NetBSD system.  The
4161.98Sapb                   image also contains tools that may be useful in repairing a
4171.98Sapb                   damaged NetBSD installation.
4181.97Stsutsui
4191.126Swiz                   Before "make install-image" is attempted, RELEASEDIR must
4201.126Swiz                   be populated by "make release" or equivalent.  The build
4211.126Swiz                   must have been performed with MKUNPRIVED=yes because "make
4221.126Swiz                   install-image" relies on information in DESTDIR/METALOG.
4231.97Stsutsui
4241.120Spgoyette     live-image    Create NetBSD live images in the RELEASEDIR/images
4251.104Swiz                   directory.  The live image contains all necessary files to
4261.104Swiz                   boot NetBSD up to multi-user mode, including all files
4271.104Swiz                   which should be extracted during installation, NetBSD
4281.104Swiz                   disklabel, bootloaders, etc.
4291.104Swiz
4301.104Swiz                   The live image is suitable for use as a disk image in
4311.104Swiz                   virtual machine environments such as QEMU, and also useful
4321.104Swiz                   to boot NetBSD from a USB flash memory stick on a real
4331.98Sapb                   machine, without the need for installation.
4341.97Stsutsui
4351.126Swiz                   Before "make live-image" is attempted, RELEASEDIR must be
4361.126Swiz                   populated by "make release" or equivalent.  The build must
4371.126Swiz                   have been performed with MKUNPRIVED=yes because "make
4381.126Swiz                   install-image" relies on information in DESTDIR/METALOG.
4391.97Stsutsui
4401.105Swiz     regression-tests
4411.20Slukem                   Can only be run after building the regression tests in the
4421.126Swiz                   directory "regress".  Runs those compiled regression tests
4431.151Slukem                   on the local host.
4441.151Slukem
4451.151Slukem                   Note: Most tests are now managed instead using atf(7); this
4461.151Slukem                   target should probably run those as well but currently does
4471.151Slukem                   not.
4481.1Stv
4491.105Swiz   The "build.sh" script
4501.96Sapb     This script file is a shell script designed to build the entire NetBSD
4511.104Swiz     system on any host with a suitable modern shell and some common
4521.105Swiz     utilities.  The required shell features are described under the HOST_SH
4531.104Swiz     variable.
4541.96Sapb
4551.96Sapb     If a host system's default shell does support the required features, then
4561.96Sapb     we suggest that you explicitly specify a suitable shell using a command
4571.96Sapb     like
4581.96Sapb
4591.105Swiz           /path/to/suitable/shell build.sh [options]
4601.96Sapb
4611.105Swiz     The above command will usually enable build.sh to automatically set
4621.105Swiz     HOST_SH=/path/to/suitable/shell, but if that fails, then the following
4631.96Sapb     set of commands may be used instead:
4641.96Sapb
4651.105Swiz           HOST_SH=/path/to/suitable/shell
4661.96Sapb           export HOST_SH
4671.105Swiz           ${HOST_SH} build.sh [options]
4681.96Sapb
4691.105Swiz     If build.sh detects that it is being executed under an unsuitable shell,
4701.148Slukem     it attempts to exec a suitable shell instead, or shows an error message.
4711.105Swiz     If HOST_SH is not set explicitly, then build.sh sets a default using
4721.96Sapb     heuristics dependent on the host platform, or from the shell under which
4731.105Swiz     build.sh is executed (if that can be determined), or using the first copy
4741.105Swiz     of sh found in PATH.
4751.1Stv
4761.1Stv     All cross-compile builds, and most native builds, of the entire system
4771.126Swiz     should make use of build.sh rather than just running "make".  This way,
4781.104Swiz     the make(1) program will be bootstrapped properly, in case the host
4791.126Swiz     system has an older or incompatible "make" program.
4801.1Stv
4811.105Swiz     When compiling the entire system via build.sh, many make(1) variables are
4821.1Stv     set for you in order to help encapsulate the build process.  In the list
4831.105Swiz     of options below, variables that are automatically set by build.sh are
4841.1Stv     noted where applicable.
4851.1Stv
4861.105Swiz     The following operations are supported by build.sh:
4871.22Slukem
4881.126Swiz     build         Build the system as per "make build".  Before the main part
4891.126Swiz                   of the build commences, this command runs the obj operation
4901.126Swiz                   (unless the -o option is given), "make cleandir" (unless
4911.126Swiz                   the -u option is given), and the tools operation.
4921.22Slukem
4931.126Swiz     distribution  Build a full distribution as per "make distribution".  This
4941.126Swiz                   command first runs the build operation.
4951.22Slukem
4961.126Swiz     release       Build a full release as per "make release".  This command
4971.105Swiz                   first runs the distribution operation.
4981.22Slukem
4991.148Slukem     help          Show a help message, and exit.
5001.148Slukem
5011.105Swiz     makewrapper   Create the nbmake-MACHINE wrapper.  This operation is
5021.104Swiz                   automatically performed for any of the other operations.
5031.23Slukem
5041.126Swiz     cleandir      Perform "make cleandir".
5051.75Slukem
5061.126Swiz     obj           Perform "make obj".
5071.22Slukem
5081.105Swiz     tools         Build and install the host tools from src/tools.  This
5091.126Swiz                   command will first run "make obj" and "make cleandir" in
5101.126Swiz                   the tools subdirectory unless the -o or -u options
5111.104Swiz                   (respectively) are given.
5121.22Slukem
5131.126Swiz     install=idir  Install the contents of DESTDIR to idir, using "make
5141.151Slukem                   installworld".
5151.151Slukem
5161.151Slukem                   Note: Files that are part of the "etc" or "xetc" sets will
5171.151Slukem                   not be installed, unless overridden by the INSTALLSETS
5181.151Slukem                   environment variable.
5191.35Slukem
5201.105Swiz     kernel=kconf  Build a new kernel.  The kconf argument is the name of a
5211.105Swiz                   configuration file suitable for use by config(1).  If kconf
5221.22Slukem                   does not contain any `/' characters, the configuration file
5231.105Swiz                   is expected to be found in the KERNCONFDIR directory, which
5241.105Swiz                   is typically sys/arch/MACHINE/conf.  The new kernel will be
5251.105Swiz                   built in a subdirectory of KERNOBJDIR, which is typically
5261.105Swiz                   sys/arch/MACHINE/compile or an associated object directory.
5271.70Sdholland
5281.105Swiz                   This command does not imply the tools command; run the
5291.105Swiz                   tools command first unless it is certain that the tools
5301.70Sdholland                   already exist and are up to date.
5311.70Sdholland
5321.126Swiz                   This command will run "make cleandir" on the kernel in
5331.105Swiz                   question first unless the -u option is given.
5341.22Slukem
5351.106Swiz     kernel.gdb=kconf
5361.106Swiz                   Build a new kernel with debug information.  Similar to the
5371.106Swiz                   above kernel=kconf operation, but creates a netbsd.gdb file
5381.106Swiz                   alongside of the kernel netbsd, which contains a full
5391.106Swiz                   symbol table and can be used for debugging (for example
5401.106Swiz                   with a cross-gdb built by MKCROSSGDB).
5411.106Swiz
5421.113Suebayasi     kernels       This command will build all kernels defined in port
5431.113Suebayasi                   specific release build procedure.
5441.111Suebayasi
5451.112Suebayasi                   This command internally calls the kernel=kconf operation
5461.112Suebayasi                   for each found kernel configuration file.
5471.111Suebayasi
5481.105Swiz     modules       This command will build kernel modules and install them
5491.105Swiz                   into DESTDIR.
5501.86Sjnemeth
5511.105Swiz     releasekernel=kconf
5521.70Sdholland                   Install a gzip(1)ed copy of the kernel previously built by
5531.105Swiz                   kernel=kconf into
5541.105Swiz                   RELEASEDIR/RELEASEMACHINEDIR/binary/kernel, usually as
5551.126Swiz                   netbsd-kconf.gz, although the "netbsd" prefix is determined
5561.126Swiz                   from the "config" directives in kconf.
5571.24Slukem
5581.126Swiz     sets          Perform "make sets".
5591.31Slukem
5601.126Swiz     sourcesets    Perform "make sourcesets".
5611.22Slukem
5621.126Swiz     syspkgs       Perform "make syspkgs".
5631.54Sapb
5641.126Swiz     iso-image     Perform "make iso-image".
5651.56Sapb
5661.105Swiz     iso-image-source
5671.126Swiz                   Perform "make iso-image-source".
5681.65Sjnemeth
5691.105Swiz     install-image
5701.126Swiz                   Perform "make install-image".
5711.97Stsutsui
5721.126Swiz     live-image    Perform "make live-image".
5731.97Stsutsui
5741.148Slukem     list-arch     Show a list of valid MACHINE and MACHINE_ARCH settings, the
5751.148Slukem                   default MACHINE_ARCH for each MACHINE, and aliases for
5761.109Sapb                   MACHINE/MACHINE_ARCH pairs, and then exits.  The -m or -a
5771.109Sapb                   options (or both) may be used to specify glob patterns that
5781.109Sapb                   will be used to narrow the list of results; for example,
5791.131Sleot                   "build.sh -m 'evb*' -a '*arm*' list-arch" will list all
5801.109Sapb                   known MACHINE/MACHINE_ARCH values in which either MACHINE
5811.109Sapb                   or ALIAS matches the pattern `evb*', and MACHINE_ARCH
5821.109Sapb                   matches the pattern `*arm*'.
5831.109Sapb
5841.105Swiz     The following command line options alter the behaviour of the build.sh
5851.51Swiz     operations described above:
5861.1Stv
5871.109Sapb     -a arch   Set the value of MACHINE_ARCH to arch.  See the -m option for
5881.109Sapb               more information.
5891.9Sthorpej
5901.105Swiz     -B buildid
5911.105Swiz               Set the value of BUILDID to buildid.  This will also append the
5921.126Swiz               build identifier to the name of the "make" wrapper script so
5931.37Slukem               that the resulting name is of the form
5941.126Swiz               "nbmake-MACHINE-BUILDID".
5951.1Stv
5961.105Swiz     -C cdextras
5971.105Swiz               Append cdextras to the CDEXTRA variable, which is a space-
5981.104Swiz               separated list of files or directories that will be added to
5991.126Swiz               the CD-ROM image that may be create by the "iso-image" or
6001.126Swiz               "iso-image-source" operations.  Files will be added to the root
6011.126Swiz               of the CD-ROM image, whereas directories will be copied
6021.87Sapb               recursively.  If relative paths are specified, they will be
6031.87Sapb               converted to absolute paths before being used.  Multiple paths
6041.105Swiz               may be specified via multiple -C options, or via a single
6051.87Sapb               option whose argument contains multiple space-separated paths.
6061.66Sjnemeth
6071.136Suwe     -c compiler
6081.136Suwe               Select the compiler for the toolchain to build NetBSD and for
6091.136Suwe               inclusion in the NetBSD distribution.  Supported choices:
6101.136Suwe
6111.136Suwe                     clang
6121.136Suwe
6131.136Suwe                     gcc [default]
6141.136Suwe
6151.136Suwe               The compiler used to build the toolchain can be different; see
6161.136Suwe               HOST_CC and HOST_CXX.
6171.136Suwe
6181.105Swiz     -D dest   Set the value of DESTDIR to dest.  If a relative path is
6191.104Swiz               specified, it will be converted to an absolute path before
6201.104Swiz               being used.
6211.104Swiz
6221.105Swiz     -E        Set `expert' mode.  This overrides various sanity checks, and
6231.105Swiz               allows: DESTDIR does not have to be set to a non-root path for
6241.105Swiz               builds, and MKUNPRIVED=yes does not have to be set when
6251.104Swiz               building as a non-root user.
6261.23Slukem
6271.105Swiz               Note: It is highly recommended that you know what you are doing
6281.23Slukem               when you use this option.
6291.19Slukem
6301.148Slukem     -h        Show a help message, and exit.
6311.51Swiz
6321.105Swiz     -j njob   Run up to njob make(1) subjobs in parallel; passed through to
6331.74Sapb               make(1).  If you see failures for reasons other than running
6341.105Swiz               out of memory while using build.sh with -j, please save
6351.104Swiz               complete build logs so the failures can be analyzed.
6361.74Sapb
6371.105Swiz               To achieve the fastest builds, -j values between (1 + the
6381.104Swiz               number of CPUs) and (2 * the number of CPUs) are recommended.
6391.104Swiz               Use lower values on machines with limited memory or I/O
6401.104Swiz               bandwidth.
6411.104Swiz
6421.126Swiz     -M obj    Set MAKEOBJDIRPREFIX to obj.  Unsets MAKEOBJDIR.  See "-O obj"
6431.126Swiz               for more information.
6441.104Swiz
6451.105Swiz               For instance, if the source directory is /usr/src, a setting of
6461.126Swiz               "-M /usr/obj" will place build-time files under
6471.105Swiz               /usr/obj/usr/src/bin, /usr/obj/usr/src/lib,
6481.105Swiz               /usr/obj/usr/src/usr.bin, and so forth.
6491.88Sapb
6501.73Sapb               If a relative path is specified, it will be converted to an
6511.105Swiz               absolute path before being used.  build.sh imposes the
6521.105Swiz               restriction that the argument to the -M option must not begin
6531.126Swiz               with a "$" (dollar sign) character; otherwise it would be too
6541.104Swiz               difficult to determine whether the value is an absolute or a
6551.104Swiz               relative path.  If the directory does not already exist,
6561.105Swiz               build.sh will create it.
6571.18Slukem
6581.109Sapb     -m mach   Set the value of MACHINE to mach, unless the mach argument is
6591.109Sapb               an alias that refers to a MACHINE/MACHINE_ARCH pair, in which
6601.109Sapb               case both MACHINE and MACHINE_ARCH are set from the alias.
6611.109Sapb               Such aliases are interpreted entirely by build.sh; they are not
6621.109Sapb               used by any other part of the build system.  The MACHINE_ARCH
6631.109Sapb               setting implied by mach will override any value of MACHINE_ARCH
6641.109Sapb               in the process environment, but will not override a value set
6651.109Sapb               by the -a option.  All cross builds require -m, but if unset on
6661.109Sapb               a NetBSD host, the host's value of MACHINE will be detected and
6671.109Sapb               used automatically.
6681.109Sapb
6691.109Sapb               See the list-arch operation for a way to get a list of valid
6701.109Sapb               MACHINE and MACHINE_ARCH settings.
6711.1Stv
6721.105Swiz     -N noiselevel
6731.126Swiz               Set the "noisyness" level of the build, by setting MAKEVERBOSE
6741.126Swiz               to noiselevel.
6751.45Slukem
6761.105Swiz     -n        Show the commands that would be executed by build.sh, but do
6771.126Swiz               not make any changes.  This is similar in concept to "make -n".
6781.1Stv
6791.105Swiz     -O obj    Create an appropriate transform macro for MAKEOBJDIR that will
6801.105Swiz               place the built object files under obj.  Unsets
6811.105Swiz               MAKEOBJDIRPREFIX.
6821.88Sapb
6831.126Swiz               For instance, a setting of "-O /usr/obj" will place build-time
6841.126Swiz               files under /usr/obj/bin, /usr/obj/lib, /usr/obj/usr.bin, and
6851.126Swiz               so forth.
6861.88Sapb
6871.88Sapb               If a relative path is specified, it will be converted to an
6881.105Swiz               absolute path before being used.  build.sh imposes the
6891.105Swiz               restriction that the argument to the -O option must not contain
6901.126Swiz               a "$" (dollar sign) character.  If the directory does not
6911.105Swiz               already exist, build.sh will create it.
6921.73Sapb
6931.105Swiz               In normal use, exactly one of the -M or -O options should be
6941.105Swiz               specified.  If neither -M nor -O is specified, then a default
6951.78Sapb               object directory will be chosen according to rules in
6961.78Sapb               <bsd.obj.mk>.  Relying on this default is not recommended
6971.78Sapb               because it is determined by complex rules that are influenced
6981.78Sapb               by the values of several variables and by the location of the
6991.78Sapb               source directory.
7001.18Slukem
7011.151Slukem               Note: Placing the obj directory location outside of the default
7021.151Slukem               source tree hierarchy makes it easier to manually clear out old
7031.151Slukem               files in the event the "make cleandir" operation is unable to
7041.151Slukem               do so.  (See CAVEATS below.)
7051.151Slukem
7061.151Slukem               Note: The use of one of -M or -O is the only means of building
7071.151Slukem               multiple machine architecture userlands from the same source
7081.151Slukem               tree without cleaning between builds (in which case, one would
7091.151Slukem               specify distinct obj locations for each).
7101.109Sapb
7111.126Swiz     -o        Set the value of MKOBJDIRS to "no".  Otherwise, it will be
7121.126Swiz               automatically set to "yes".  This default is opposite to the
7131.105Swiz               behaviour when not using build.sh.
7141.1Stv
7151.148Slukem     -P        Set the value of MKREPRO and MKREPRO_TIMESTAMP to the latest
7161.148Slukem               source CVS timestamp for reproducible builds.
7171.148Slukem
7181.105Swiz     -R rel    Set the value of RELEASEDIR to rel.  If a relative path is
7191.59Sapb               specified, it will be converted to an absolute path before
7201.59Sapb               being used.
7211.18Slukem
7221.105Swiz     -r        Remove the contents of DESTDIR and TOOLDIR before building
7231.1Stv               (provides a clean starting point).  This will skip deleting
7241.105Swiz               DESTDIR if building on a native system to the root directory.
7251.1Stv
7261.105Swiz     -S seed   Change the value of BUILDSEED to seed.  This should rarely be
7271.72Sperry               necessary.
7281.72Sperry
7291.105Swiz     -T tools  Set the value of TOOLDIR to tools.  If a relative path is
7301.104Swiz               specified, it will be converted to an absolute path before
7311.126Swiz               being used.  If set, the bootstrap "make" will only be rebuilt
7321.126Swiz               if the source files for make(1) have changed.
7331.104Swiz
7341.105Swiz     -U        Set MKUNPRIVED=yes.
7351.104Swiz
7361.105Swiz     -u        Set MKUPDATE=yes.
7371.104Swiz
7381.105Swiz     -V var=[value]
7391.105Swiz               Set the environment variable var to an optional value.  This is
7401.105Swiz               propagated to the nbmake wrapper.
7411.105Swiz
7421.105Swiz     -w wrapper
7431.105Swiz               Create the nbmake wrapper script (see below) in a custom
7441.105Swiz               location, specified by wrapper.  This allows, for instance, to
7451.151Slukem               place the wrapper in PATH automatically.
7461.151Slukem
7471.151Slukem               Note: wrapper is the full name of the file, not just a
7481.151Slukem               directory name.  If a relative path is specified, it will be
7491.151Slukem               converted to an absolute path before being used.
7501.1Stv
7511.105Swiz     -X x11src
7521.105Swiz               Set the value of X11SRCDIR to x11src.  If a relative path is
7531.59Sapb               specified, it will be converted to an absolute path before
7541.59Sapb               being used.
7551.49Slukem
7561.105Swiz     -x        Set MKX11=yes.
7571.49Slukem
7581.105Swiz     -Z var    Unset ("zap") the environment variable var.  This is propagated
7591.105Swiz               to the nbmake wrapper.
7601.40Slukem
7611.148Slukem     -?        Show a help message, and exit.
7621.148Slukem
7631.105Swiz   The "nbmake-MACHINE" wrapper script
7641.105Swiz     If using the build.sh script to build NetBSD, a nbmake-MACHINE script
7651.105Swiz     will be created in TOOLDIR/bin upon the first build to assist in building
7661.1Stv     subtrees on a cross-compile host.
7671.1Stv
7681.105Swiz     nbmake-MACHINE can be invoked in lieu of make(1), and will instead call
7691.126Swiz     the up-to-date version of "nbmake" installed into TOOLDIR/bin with
7701.105Swiz     several key variables pre-set, including MACHINE, MACHINE_ARCH, and
7711.105Swiz     TOOLDIR.  nbmake-MACHINE will also set variables specified with -V, and
7721.105Swiz     unset variables specified with -Z.
7731.40Slukem
7741.105Swiz     This script can be symlinked into a directory listed in PATH, or called
7751.1Stv     with an absolute path.
7761.1Stv
7771.105SwizEXAMPLES
7781.151Slukem     1.   % ./build.sh [OPTIONS] tools kernel=GENERIC
7791.28Slukem
7801.28Slukem          Build a new toolchain, and use the new toolchain to configure and
7811.28Slukem          build a new GENERIC kernel.
7821.28Slukem
7831.151Slukem     2.   % ./build.sh [OPTIONS] -U distribution
7841.28Slukem
7851.105Swiz          Using unprivileged mode, build a complete distribution to a DESTDIR
7861.148Slukem          directory that build.sh selects (and will show).
7871.28Slukem
7881.151Slukem     3.   # ./build.sh [OPTIONS] -U install=/
7891.28Slukem
7901.105Swiz          As root, install to / the distribution that was built by example 2.
7911.105Swiz          Even though this is run as root, -U is required so that the
7921.105Swiz          permissions stored in DESTDIR/METALOG are correctly applied to the
7931.105Swiz          files as they're copied to /.
7941.28Slukem
7951.151Slukem     4.   % ./build.sh [OPTIONS] -U -u release
7961.28Slukem
7971.105Swiz          Using unprivileged mode, build a complete release to DESTDIR and
7981.148Slukem          RELEASEDIR directories that build.sh selects (and will show).
7991.126Swiz          MKUPDATE=yes (-u) is set to prevent the "make cleandir", so that if
8001.126Swiz          this is run after example 2, it doesn't need to redo that portion of
8011.126Swiz          the release build.
8021.16Slukem
8031.105SwizSEE ALSO
8041.155Slukem     config(1), ctags(1), ex(1), gzip(1), lint(1), make(1), mandoc(1),
8051.155Slukem     mkisofs(1), nroff(1), vi(1), mk.conf(5), atf(7), hier(7), mdoc(7),
8061.155Slukem     release(7), etcupdate(8), installboot(8), mount(8), postinstall(8),
8071.151Slukem     sysinst(8), pkgsrc/sysutils/cdrtools
8081.1Stv
8091.155Slukem     Note: The NetBSD manual pages are also available at
8101.155Slukem     https://man.netbsd.org
8111.155Slukem
8121.105SwizHISTORY
8131.105Swiz     The build.sh based build scheme was introduced for NetBSD 1.6 as
8141.105Swiz     USE_NEW_TOOLCHAIN, and re-worked to TOOLCHAIN_MISSING after that.
8151.1Stv
8161.109SapbCAVEATS
8171.109Sapb     After significant updates to third-party components in the source tree,
8181.126Swiz     the "make cleandir" operation may be insufficient to clean out old files
8191.126Swiz     in object directories.  Instead, one may have to manually remove the
8201.126Swiz     files.  Consult the UPDATING file for notices concerning this.
8211.109Sapb
8221.156SlukemNetBSD                           July 5, 2023                           NetBSD
823