BUILDING revision 1.157
11.104SwizBUILDING(8)                 System Manager's Manual                BUILDING(8)
21.1Stv
31.105SwizNAME
41.150Slukem     BUILDING - Procedure for building NetBSD from source code
51.1Stv
61.105SwizREQUIREMENTS
71.1Stv     NetBSD is designed to be buildable on most POSIX-compliant host systems.
81.105Swiz     The basic build procedure is the same whether compiling natively (on the
91.105Swiz     same NetBSD architecture) or cross compiling (on another architecture or
101.1Stv     OS).
111.1Stv
121.157Slukem     This source tree contains the build.sh shell script which supports both
131.157Slukem     native and cross builds of NetBSD.
141.157Slukem
151.126Swiz     This source tree contains a special subtree, "tools", which uses the host
161.126Swiz     system to create a build toolchain for the target architecture.  The host
171.126Swiz     system must have at least C and C++ compilers in order to create the
181.157Slukem     toolchain (make(1) is not required); all other tools (including make(1)
191.157Slukem     as nbmake) are created as part of the NetBSD build process.  (See the
201.157Slukem     Environment variables section below if you need to override or manually
211.157Slukem     select your compilers.)
221.1Stv
231.155Slukem     Note: Within this document, cross-references to manual pages are to the
241.155Slukem     NetBSD manual pages, not the host system manual pages.  The mdoc(7)
251.155Slukem     source to the NetBSD manual pages can be found within the source tree,
261.155Slukem     and these and can be formatted with mandoc(1) or nroff(1) if those are
271.155Slukem     available on the host system.  The NetBSD manual pages are also available
281.155Slukem     at https://man.netbsd.org
291.155Slukem
301.105SwizFILES
311.105Swiz   Source tree layout
321.151Slukem     BUILDING       This document (in plaintext).  Generated from
331.151Slukem                    doc/BUILDING.mdoc.
341.48Sjmc
351.105Swiz     Makefile       The main Makefile for NetBSD; should only be run for
361.44Slukem                    native builds with an appropriately up-to-date version of
371.142Sandvar                    NetBSD make(1).  Intended for expert use with knowledge of
381.129Ssevan                    its shortcomings, it has been superseded by the build.sh
391.129Ssevan                    shell script as the recommended means for building NetBSD.
401.1Stv
411.105Swiz     UPDATING       Special notes for updating from an earlier revision of
421.1Stv                    NetBSD.  It is important to read this file before every
431.1Stv                    build of an updated source tree.
441.1Stv
451.105Swiz     build.sh       Bourne-compatible shell script used for building the host
461.1Stv                    build tools and the NetBSD system from scratch.  Can be
471.1Stv                    used for both native and cross builds, and should be used
481.129Ssevan                    instead of make(1) as it performs additional checks to
491.129Ssevan                    prevent common issues going undetected, such as building
501.129Ssevan                    with an outdated version of make(1).
511.1Stv
521.105Swiz     crypto/dist/, dist/, gnu/dist/
531.104Swiz                    Sources imported verbatim from third parties, without
541.104Swiz                    mangling the existing build structure.  Other source trees
551.126Swiz                    in bin through usr.sbin use the NetBSD make(1) "reachover"
561.126Swiz                    Makefile semantics when building these programs for a
571.126Swiz                    native host.
581.104Swiz
591.151Slukem     distrib/, etc/
601.151Slukem                    Sources for items used when making a full release
611.151Slukem                    snapshot, such as files installed in DESTDIR/etc on the
621.151Slukem                    destination system, boot media, and release notes.
631.151Slukem
641.151Slukem     doc/BUILDING.mdoc
651.155Slukem                    The source to this document, in mdoc(7) format.  Used to
661.155Slukem                    generate BUILDING.
671.151Slukem
681.106Swiz     external, sys/external
691.106Swiz                    Sources and build infrastructure for components imported
701.106Swiz                    (mostly) unchanged from upstream maintainers, sorted by
711.106Swiz                    applicable license.  This is (slowly) replacing the
721.106Swiz                    crypto/dist, dist, and gnu/dist directories.
731.106Swiz
741.151Slukem     external/mit/xorg/
751.151Slukem                    "Reachover" build structure for modular Xorg; the source
761.151Slukem                    is in X11SRCDIR.
771.1Stv
781.157Slukem     mk.conf        Optional source tree specific mk.conf(5), used (if
791.157Slukem                    present) instead of /etc/mk.conf unless MAKECONF is
801.157Slukem                    defined.
811.157Slukem
821.157Slukem                    Note: Not part of the NetBSD source repository.
831.157Slukem
841.151Slukem     regress/, tests/
851.70Sdholland                    Regression test harness.  Can be cross-compiled, but only
861.105Swiz                    run natively.  tests/ uses the atf(7) test framework;
871.105Swiz                    regress/ contains older tests that have not yet been
881.70Sdholland                    migrated to atf(7).
891.1Stv
901.105Swiz     sys/           NetBSD kernel sources.
911.1Stv
921.126Swiz     tools/         "Reachover" build structure for the host build tools.
931.104Swiz                    This has a special method of determining out-of-date
941.104Swiz                    status.
951.1Stv
961.151Slukem     tools/compat/README
971.151Slukem                    Special notes for cross-hosting a NetBSD build on non-
981.151Slukem                    NetBSD platforms.
991.151Slukem
1001.151Slukem     Other directories including bin/ ... usr.sbin/
1011.1Stv                    Sources to the NetBSD userland (non-kernel) programs.  If
1021.1Stv                    any of these directories are missing, they will be skipped
1031.1Stv                    during the build.
1041.1Stv
1051.105Swiz   Build tree layout
1061.156Slukem     The NetBSD build tree is described in hier(7) (whose mdoc(7) source is in
1071.155Slukem     share/man/man7/hier.7), and the release layout is described in release(7)
1081.156Slukem     (whose mdoc(7) source is in share/man/man7/release.7).
1091.1Stv
1101.105SwizCONFIGURATION
1111.105Swiz   Environment variables
1121.8Slukem     Several environment variables control the behaviour of NetBSD builds.
1131.8Slukem
1141.152Slukem     HOST_CC         Path name to C compiler used to create the toolchain.
1151.58Sapb
1161.155Slukem                     Default: "cc".
1171.155Slukem
1181.152Slukem     HOST_CFLAGS     Flags passed to the host C compiler.
1191.53Sreed
1201.155Slukem                     Default: "-O".
1211.155Slukem
1221.155Slukem     HOST_CPPFLAGS   Flags passed to the host C/C++ pre-processor.
1231.155Slukem
1241.155Slukem                     Default: Unset.
1251.155Slukem
1261.152Slukem     HOST_CXX        Path name to C++ compiler used to create the toolchain.
1271.137Suwe
1281.155Slukem                     Default: Unset, but defaults to "c++" where required.
1291.155Slukem
1301.152Slukem     HOST_CXXFLAGS   Flags passed to the host C++ compiler.
1311.53Sreed
1321.155Slukem                     Default: Unset.
1331.155Slukem
1341.152Slukem     HOST_SH         Path name to a shell available on the host system and
1351.152Slukem                     suitable for use during the build.  The NetBSD build
1361.152Slukem                     system requires a modern Bourne-like shell with POSIX-
1371.152Slukem                     compliant features, and also requires support for the
1381.152Slukem                     "local" keyword to declare local variables in shell
1391.152Slukem                     functions (which is a widely-implemented but non-
1401.152Slukem                     standardised feature).
1411.152Slukem
1421.152Slukem                     Depending on the host system, a suitable shell may be
1431.152Slukem                     /bin/sh, /usr/xpg4/bin/sh, /bin/ksh (provided it is a
1441.152Slukem                     variant of ksh that supports the "local" keyword, such as
1451.152Slukem                     ksh88, but not ksh93), or /usr/local/bin/bash.
1461.152Slukem
1471.152Slukem                     Most parts of the build require HOST_SH to be an absolute
1481.152Slukem                     path; however, build.sh allows it to be a simple command
1491.152Slukem                     name, which will be converted to an absolute path by
1501.152Slukem                     searching the PATH.
1511.137Suwe
1521.155Slukem                     Default: "sh".
1531.155Slukem
1541.147Slukem     INSTALLBOOT_UBOOT_PATHS
1551.152Slukem                     A colon-separated list of search paths used by
1561.152Slukem                     installboot(8) to find U-Boot packages.
1571.147Slukem
1581.155Slukem                     Default: Unset.
1591.155Slukem
1601.152Slukem     MACHINE         Machine type, e.g., "macppc".
1611.8Slukem
1621.155Slukem                     Default: Unset.
1631.155Slukem
1641.152Slukem     MACHINE_ARCH    Machine architecture, e.g., "powerpc".
1651.8Slukem
1661.155Slukem                     Default: Unset.
1671.155Slukem
1681.152Slukem     MAKE            Path name to invoke make(1) as.
1691.8Slukem
1701.155Slukem                     Default: "make".
1711.155Slukem
1721.152Slukem     MAKECONF        The name of the make(1) configuration file.  See "make"
1731.152Slukem                     variables and mk.conf(5).
1741.151Slukem
1751.152Slukem                     Note: Only settable in the process environment.
1761.151Slukem
1771.157Slukem                     Default: "/etc/mk.conf", although build.sh will set the
1781.157Slukem                     default to the full path to mk.conf if the latter is
1791.157Slukem                     present in the same directory as build.sh.
1801.151Slukem
1811.152Slukem     MAKEFLAGS       Flags to invoke make(1) with.
1821.151Slukem
1831.152Slukem                     Note: build.sh ignores the value of MAKEFLAGS passed in
1841.152Slukem                     the environment, but allows MAKEFLAGS to be set via the
1851.152Slukem                     -V option.
1861.151Slukem
1871.155Slukem                     Default: "-X" on systems with a small ARG_MAX (Cygwin,
1881.155Slukem                     Darwin, FreeBSD); otherwise unset.
1891.155Slukem
1901.152Slukem     MAKEOBJDIR      Directory to use as the .OBJDIR for the current
1911.152Slukem                     directory.  The value is subjected to variable expansion
1921.152Slukem                     by make(1).  Typical usage is to set this variable to a
1931.152Slukem                     value involving the use of `${.CURDIR:S...}' or
1941.152Slukem                     `${.CURDIR:C...}', to derive the value of .OBJDIR from
1951.152Slukem                     the value of .CURDIR.  Used only if MAKEOBJDIRPREFIX is
1961.152Slukem                     not defined.
1971.151Slukem
1981.152Slukem                     Note: MAKEOBJDIR can be provided only in the environment
1991.152Slukem                     or via the -O flag of build.sh; it cannot usefully be set
2001.152Slukem                     inside a Makefile, including in mk.conf(5) or MAKECONF.
2011.151Slukem
2021.155Slukem                     Default: Unset.
2031.155Slukem
2041.151Slukem     MAKEOBJDIRPREFIX
2051.152Slukem                     Top level directory of the object directory tree.  The
2061.152Slukem                     value is subjected to variable expansion by make(1).
2071.152Slukem                     build.sh will create the ${MAKEOBJDIRPREFIX} directory if
2081.152Slukem                     necessary, but if make(1) is used without build.sh, then
2091.152Slukem                     rules in <bsd.obj.mk> will abort the build if the
2101.152Slukem                     ${MAKEOBJDIRPREFIX} directory does not exist.  If the
2111.152Slukem                     value is defined and valid, then
2121.152Slukem                     ${MAKEOBJDIRPREFIX}/${.CURDIR} is used as the .OBJDIR for
2131.152Slukem                     the current directory.  The current directory may be read
2141.152Slukem                     only.
2151.152Slukem
2161.152Slukem                     Note: MAKEOBJDIRPREFIX can be provided only in the
2171.152Slukem                     environment or via the -M flag of build.sh; it cannot
2181.152Slukem                     usefully be set inside a Makefile, including in
2191.152Slukem                     mk.conf(5) or MAKECONF.
2201.152Slukem
2211.155Slukem                     Default: Unset.
2221.155Slukem
2231.152Slukem     TMPDIR          Top-level directory to store temporary directories used
2241.152Slukem                     by build.sh before paths to other directories such as
2251.152Slukem                     .OBJDIR can be determined.
2261.149Slukem
2271.152Slukem                     Note: Must support execution of binaries.  I.e., without
2281.152Slukem                     mount(8)'s -o noexec option.
2291.149Slukem
2301.152Slukem                     Default: "/tmp".
2311.149Slukem
2321.105Swiz   "make" variables
2331.156Slukem     Variables that control the behavior of NetBSD builds are documented in
2341.156Slukem     mk.conf(5) (whose mdoc(7) source is in share/man/man5/mk.conf.5).
2351.45Slukem
2361.155Slukem     Unless otherwise specified, these variables may be set in either the
2371.155Slukem     process environment or the make(1) configuration file mk.conf(5)
2381.155Slukem     specified by MAKECONF.
2391.49Slukem
2401.105SwizBUILDING
2411.105Swiz   "make" command line options
2421.52Sjmc     This is not a summary of all the options available to make(1); only the
2431.52Sjmc     options used most frequently with NetBSD builds are listed here.
2441.1Stv
2451.105Swiz     -j njob    Run up to njob make(1) subjobs in parallel.  Makefiles should
2461.52Sjmc                use .WAIT or have explicit dependencies as necessary to
2471.74Sapb                enforce build ordering.
2481.10Ssommerfe
2491.105Swiz     -m dir     Specify the default directory for searching for system
2501.104Swiz                Makefile segments, mainly the <bsd.*.mk> files.  When building
2511.104Swiz                any full NetBSD source tree, this should be set to the
2521.126Swiz                "share/mk" directory in the source tree.  This is set
2531.104Swiz                automatically when building from the top level, or when using
2541.105Swiz                build.sh.
2551.1Stv
2561.148Slukem     -n         Show the commands that would have been executed, but do not
2571.1Stv                actually execute them.  This will still cause recursion to
2581.1Stv                take place.
2591.1Stv
2601.148Slukem     -V var     Show make(1)'s idea of the value of var.  Does not build any
2611.1Stv                targets.
2621.1Stv
2631.105Swiz     var=value  Set the variable var to value, overriding any setting
2641.105Swiz                specified by the process environment, the MAKECONF
2651.104Swiz                configuration file, or the system Makefile segments.
2661.1Stv
2671.105Swiz   "make" targets
2681.1Stv     These default targets may be built by running make(1) in any subtree of
2691.1Stv     the NetBSD source code.  It is recommended that none of these be used
2701.126Swiz     from the top level Makefile; as a specific exception, "make obj" and
2711.126Swiz     "make cleandir" are useful in that context.
2721.1Stv
2731.105Swiz     all        Build programs, libraries, and preformatted documentation.
2741.1Stv
2751.105Swiz     clean      Remove program and library object code files.
2761.1Stv
2771.105Swiz     cleandir   Same as clean, but also remove preformatted documentation,
2781.126Swiz                dependency files generated by "make depend", and any other
2791.49Slukem                files known to be created at build time.
2801.1Stv
2811.105Swiz     depend     Create dependency files (.depend) containing more detailed
2821.44Slukem                information about the dependencies of source code on header
2831.1Stv                files.  Allows programs to be recompiled automatically when a
2841.1Stv                dependency changes.
2851.1Stv
2861.126Swiz     dependall  Does a "make depend" immediately followed by a "make all".
2871.10Ssommerfe                This improves cache locality of the build since both passes
2881.10Ssommerfe                read the source files in their entirety.
2891.1Stv
2901.105Swiz     distclean  Synonym for cleandir.
2911.49Slukem
2921.105Swiz     includes   Build and install system header files.  Typically needed
2931.44Slukem                before any system libraries or programs can be built.
2941.1Stv
2951.105Swiz     install    Install programs, libraries, and documentation into DESTDIR.
2961.105Swiz                Few files will be installed to DESTDIR/dev, DESTDIR/etc,
2971.105Swiz                DESTDIR/root or DESTDIR/var in order to prevent user supplied
2981.27Slukem                configuration data from being overwritten.
2991.1Stv
3001.105Swiz     lint       Run lint(1) against the C source code, where appropriate, and
3011.1Stv                generate system-installed lint libraries.
3021.1Stv
3031.105Swiz     obj        Create object directories to be used for built files, instead
3041.1Stv                of building directly in the source tree.
3051.1Stv
3061.105Swiz     tags       Create ctags(1) searchable function lists usable by the ex(1)
3071.1Stv                and vi(1) text editors.
3081.1Stv
3091.105Swiz   "make" targets for the top level
3101.1Stv     Additional make(1) targets are usable specifically from the top source
3111.1Stv     level to facilitate building the entire NetBSD source tree.
3121.1Stv
3131.105Swiz     build         Build the entire NetBSD system (except the kernel).  This
3141.68Sreed                   orders portions of the source tree such that prerequisites
3151.68Sreed                   will be built in the proper order.
3161.20Slukem
3171.126Swiz     distribution  Do a "make build", and then install a full distribution
3181.105Swiz                   (which does not include a kernel) into DESTDIR, including
3191.105Swiz                   files in DESTDIR/dev, DESTDIR/etc, DESTDIR/root and
3201.105Swiz                   DESTDIR/var.
3211.20Slukem
3221.126Swiz     buildworld    As per "make distribution", except that it ensures that
3231.105Swiz                   DESTDIR is not the root directory.
3241.20Slukem
3251.105Swiz     installworld  Install the distribution from DESTDIR to INSTALLWORLDDIR,
3261.70Sdholland                   which defaults to the root directory.  Ensures that
3271.105Swiz                   INSTALLWORLDDIR is not the root directory if cross
3281.104Swiz                   compiling.
3291.20Slukem
3301.105Swiz                   The INSTALLSETS environment variable may be set to a space-
3311.95Sapb                   separated list of distribution sets to be installed.  By
3321.126Swiz                   default, all sets except "etc" and "xetc" are installed, so
3331.126Swiz                   most files in INSTALLWORLDDIR/etc will not be installed or
3341.126Swiz                   modified.
3351.61Sapb
3361.105Swiz                   Note: Before performing this operation with
3371.105Swiz                   INSTALLWORLDDIR=/, it is highly recommended that you
3381.61Sapb                   upgrade your kernel and reboot.  After performing this
3391.61Sapb                   operation, it is recommended that you use etcupdate(8) to
3401.105Swiz                   update files in INSTALLWORLDDIR/etc, and postinstall(8) to
3411.102Sabs                   check for or fix inconsistencies.
3421.20Slukem
3431.105Swiz     sets          Create distribution sets from DESTDIR into
3441.105Swiz                   RELEASEDIR/RELEASEMACHINEDIR/binary/sets.  Should be run
3451.126Swiz                   after "make distribution", as "make build" alone does not
3461.126Swiz                   install all of the required files.
3471.25Slukem
3481.105Swiz     sourcesets    Create source sets of the source tree into
3491.105Swiz                   RELEASEDIR/source/sets.
3501.31Slukem
3511.105Swiz     syspkgs       Create syspkgs from DESTDIR into
3521.105Swiz                   RELEASEDIR/RELEASEMACHINEDIR/binary/syspkgs.  Should be run
3531.126Swiz                   after "make distribution", as "make build" alone does not
3541.126Swiz                   install all of the required files.
3551.54Sapb
3561.126Swiz     release       Do a "make distribution", build kernels, distribution
3571.126Swiz                   media, and install sets (this as per "make sets"), and then
3581.126Swiz                   package the system into a standard release layout as
3591.105Swiz                   described by release(7).  This requires that RELEASEDIR be
3601.25Slukem                   set (see above).
3611.1Stv
3621.105Swiz     iso-image     Create a NetBSD installation CD-ROM image in the
3631.114Ssnj                   RELEASEDIR/images directory.  The CD-ROM file system will
3641.114Ssnj                   have a layout as described in release(7).
3651.62Sapb
3661.62Sapb                   For most machine types, the CD-ROM will be bootable, and
3671.104Swiz                   will automatically run the sysinst(8) menu-based
3681.104Swiz                   installation program, which can be used to install or
3691.104Swiz                   upgrade a NetBSD system.  Bootable CD-ROMs also contain
3701.104Swiz                   tools that may be useful in repairing a damaged NetBSD
3711.104Swiz                   installation.
3721.62Sapb
3731.126Swiz                   Before "make iso-image" is attempted, RELEASEDIR must be
3741.126Swiz                   populated by "make release" or equivalent.
3751.62Sapb
3761.151Slukem                   Note: Other, smaller, CD-ROM images may be created in the
3771.151Slukem                   RELEASEDIR/RELEASEMACHINEDIR/installation/cdrom directory
3781.151Slukem                   by "make release".  These smaller images usually contain
3791.151Slukem                   the same tools as the larger images in RELEASEDIR/images,
3801.151Slukem                   but do not contain additional content such as the
3811.151Slukem                   distribution sets.
3821.62Sapb
3831.151Slukem                   Note: The mac68k port still uses an older method of
3841.104Swiz                   creating CD-ROM images.  This requires the mkisofs(1)
3851.104Swiz                   utility, which is not part of NetBSD, but which can be
3861.105Swiz                   installed from pkgsrc/sysutils/cdrtools.
3871.56Sapb
3881.105Swiz     iso-image-source
3891.64Sjnemeth                   Create a NetBSD installation CD-ROM image in the
3901.114Ssnj                   RELEASEDIR/images directory.  The CD-ROM file system will
3911.114Ssnj                   have a layout as described in release(7).  It will have top
3921.64Sjnemeth                   level directories for the machine type and source.
3931.64Sjnemeth
3941.64Sjnemeth                   For most machine types, the CD-ROM will be bootable, and
3951.104Swiz                   will automatically run the sysinst(8) menu-based
3961.104Swiz                   installation program, which can be used to install or
3971.104Swiz                   upgrade a NetBSD system.  Bootable CD-ROMs also contain
3981.104Swiz                   tools that may be useful in repairing a damaged NetBSD
3991.104Swiz                   installation.
4001.64Sjnemeth
4011.126Swiz                   Before "make iso-image-source" is attempted, RELEASEDIR
4021.126Swiz                   must be populated by "make sourcesets release" or
4031.104Swiz                   equivalent.
4041.64Sjnemeth
4051.151Slukem                   Note: Other, smaller, CD-ROM images may be created in the
4061.151Slukem                   RELEASEDIR/RELEASEMACHINEDIR/installation/cdrom directory
4071.151Slukem                   by "make release".  These smaller images usually contain
4081.151Slukem                   the same tools as the larger images in RELEASEDIR/images,
4091.151Slukem                   but do not contain additional content such as the
4101.151Slukem                   distribution sets.
4111.64Sjnemeth
4121.151Slukem                   Note: The mac68k port still uses an older method of
4131.104Swiz                   creating CD-ROM images.  This requires the mkisofs(1)
4141.104Swiz                   utility, which is not part of NetBSD, but which can be
4151.105Swiz                   installed from pkgsrc/sysutils/cdrtools.
4161.64Sjnemeth
4171.105Swiz     install-image
4181.98Sapb                   Create a bootable NetBSD installation disk image in the
4191.127Splunky                   RELEASEDIR/images directory.  The installation disk image
4201.127Splunky                   is suitable for copying to bootable USB flash memory
4211.127Splunky                   sticks, etc., for machines which are able to boot from such
4221.127Splunky                   devices.  The file system in the bootable disk image will
4231.127Splunky                   have a layout as described in release(7).
4241.97Stsutsui
4251.97Stsutsui                   The installation image is bootable, and will automatically
4261.97Stsutsui                   run the sysinst(8) menu-based installation program, which
4271.98Sapb                   can be used to install or upgrade a NetBSD system.  The
4281.98Sapb                   image also contains tools that may be useful in repairing a
4291.98Sapb                   damaged NetBSD installation.
4301.97Stsutsui
4311.126Swiz                   Before "make install-image" is attempted, RELEASEDIR must
4321.126Swiz                   be populated by "make release" or equivalent.  The build
4331.126Swiz                   must have been performed with MKUNPRIVED=yes because "make
4341.126Swiz                   install-image" relies on information in DESTDIR/METALOG.
4351.97Stsutsui
4361.120Spgoyette     live-image    Create NetBSD live images in the RELEASEDIR/images
4371.104Swiz                   directory.  The live image contains all necessary files to
4381.104Swiz                   boot NetBSD up to multi-user mode, including all files
4391.104Swiz                   which should be extracted during installation, NetBSD
4401.104Swiz                   disklabel, bootloaders, etc.
4411.104Swiz
4421.104Swiz                   The live image is suitable for use as a disk image in
4431.104Swiz                   virtual machine environments such as QEMU, and also useful
4441.104Swiz                   to boot NetBSD from a USB flash memory stick on a real
4451.98Sapb                   machine, without the need for installation.
4461.97Stsutsui
4471.126Swiz                   Before "make live-image" is attempted, RELEASEDIR must be
4481.126Swiz                   populated by "make release" or equivalent.  The build must
4491.126Swiz                   have been performed with MKUNPRIVED=yes because "make
4501.126Swiz                   install-image" relies on information in DESTDIR/METALOG.
4511.97Stsutsui
4521.105Swiz     regression-tests
4531.20Slukem                   Can only be run after building the regression tests in the
4541.126Swiz                   directory "regress".  Runs those compiled regression tests
4551.151Slukem                   on the local host.
4561.151Slukem
4571.151Slukem                   Note: Most tests are now managed instead using atf(7); this
4581.151Slukem                   target should probably run those as well but currently does
4591.151Slukem                   not.
4601.1Stv
4611.105Swiz   The "build.sh" script
4621.96Sapb     This script file is a shell script designed to build the entire NetBSD
4631.104Swiz     system on any host with a suitable modern shell and some common
4641.105Swiz     utilities.  The required shell features are described under the HOST_SH
4651.104Swiz     variable.
4661.96Sapb
4671.96Sapb     If a host system's default shell does support the required features, then
4681.96Sapb     we suggest that you explicitly specify a suitable shell using a command
4691.96Sapb     like
4701.96Sapb
4711.105Swiz           /path/to/suitable/shell build.sh [options]
4721.96Sapb
4731.105Swiz     The above command will usually enable build.sh to automatically set
4741.105Swiz     HOST_SH=/path/to/suitable/shell, but if that fails, then the following
4751.96Sapb     set of commands may be used instead:
4761.96Sapb
4771.105Swiz           HOST_SH=/path/to/suitable/shell
4781.96Sapb           export HOST_SH
4791.105Swiz           ${HOST_SH} build.sh [options]
4801.96Sapb
4811.105Swiz     If build.sh detects that it is being executed under an unsuitable shell,
4821.148Slukem     it attempts to exec a suitable shell instead, or shows an error message.
4831.105Swiz     If HOST_SH is not set explicitly, then build.sh sets a default using
4841.96Sapb     heuristics dependent on the host platform, or from the shell under which
4851.105Swiz     build.sh is executed (if that can be determined), or using the first copy
4861.105Swiz     of sh found in PATH.
4871.1Stv
4881.1Stv     All cross-compile builds, and most native builds, of the entire system
4891.126Swiz     should make use of build.sh rather than just running "make".  This way,
4901.104Swiz     the make(1) program will be bootstrapped properly, in case the host
4911.126Swiz     system has an older or incompatible "make" program.
4921.1Stv
4931.105Swiz     When compiling the entire system via build.sh, many make(1) variables are
4941.1Stv     set for you in order to help encapsulate the build process.  In the list
4951.105Swiz     of options below, variables that are automatically set by build.sh are
4961.1Stv     noted where applicable.
4971.1Stv
4981.105Swiz     The following operations are supported by build.sh:
4991.22Slukem
5001.126Swiz     build         Build the system as per "make build".  Before the main part
5011.126Swiz                   of the build commences, this command runs the obj operation
5021.126Swiz                   (unless the -o option is given), "make cleandir" (unless
5031.126Swiz                   the -u option is given), and the tools operation.
5041.22Slukem
5051.126Swiz     distribution  Build a full distribution as per "make distribution".  This
5061.126Swiz                   command first runs the build operation.
5071.22Slukem
5081.126Swiz     release       Build a full release as per "make release".  This command
5091.105Swiz                   first runs the distribution operation.
5101.22Slukem
5111.148Slukem     help          Show a help message, and exit.
5121.148Slukem
5131.105Swiz     makewrapper   Create the nbmake-MACHINE wrapper.  This operation is
5141.104Swiz                   automatically performed for any of the other operations.
5151.23Slukem
5161.126Swiz     cleandir      Perform "make cleandir".
5171.75Slukem
5181.126Swiz     obj           Perform "make obj".
5191.22Slukem
5201.105Swiz     tools         Build and install the host tools from src/tools.  This
5211.126Swiz                   command will first run "make obj" and "make cleandir" in
5221.126Swiz                   the tools subdirectory unless the -o or -u options
5231.104Swiz                   (respectively) are given.
5241.22Slukem
5251.126Swiz     install=idir  Install the contents of DESTDIR to idir, using "make
5261.151Slukem                   installworld".
5271.151Slukem
5281.151Slukem                   Note: Files that are part of the "etc" or "xetc" sets will
5291.151Slukem                   not be installed, unless overridden by the INSTALLSETS
5301.151Slukem                   environment variable.
5311.35Slukem
5321.105Swiz     kernel=kconf  Build a new kernel.  The kconf argument is the name of a
5331.105Swiz                   configuration file suitable for use by config(1).  If kconf
5341.22Slukem                   does not contain any `/' characters, the configuration file
5351.105Swiz                   is expected to be found in the KERNCONFDIR directory, which
5361.105Swiz                   is typically sys/arch/MACHINE/conf.  The new kernel will be
5371.105Swiz                   built in a subdirectory of KERNOBJDIR, which is typically
5381.105Swiz                   sys/arch/MACHINE/compile or an associated object directory.
5391.70Sdholland
5401.105Swiz                   This command does not imply the tools command; run the
5411.105Swiz                   tools command first unless it is certain that the tools
5421.70Sdholland                   already exist and are up to date.
5431.70Sdholland
5441.126Swiz                   This command will run "make cleandir" on the kernel in
5451.105Swiz                   question first unless the -u option is given.
5461.22Slukem
5471.106Swiz     kernel.gdb=kconf
5481.106Swiz                   Build a new kernel with debug information.  Similar to the
5491.106Swiz                   above kernel=kconf operation, but creates a netbsd.gdb file
5501.106Swiz                   alongside of the kernel netbsd, which contains a full
5511.106Swiz                   symbol table and can be used for debugging (for example
5521.106Swiz                   with a cross-gdb built by MKCROSSGDB).
5531.106Swiz
5541.113Suebayasi     kernels       This command will build all kernels defined in port
5551.113Suebayasi                   specific release build procedure.
5561.111Suebayasi
5571.112Suebayasi                   This command internally calls the kernel=kconf operation
5581.112Suebayasi                   for each found kernel configuration file.
5591.111Suebayasi
5601.105Swiz     modules       This command will build kernel modules and install them
5611.105Swiz                   into DESTDIR.
5621.86Sjnemeth
5631.105Swiz     releasekernel=kconf
5641.70Sdholland                   Install a gzip(1)ed copy of the kernel previously built by
5651.105Swiz                   kernel=kconf into
5661.105Swiz                   RELEASEDIR/RELEASEMACHINEDIR/binary/kernel, usually as
5671.126Swiz                   netbsd-kconf.gz, although the "netbsd" prefix is determined
5681.126Swiz                   from the "config" directives in kconf.
5691.24Slukem
5701.126Swiz     sets          Perform "make sets".
5711.31Slukem
5721.126Swiz     sourcesets    Perform "make sourcesets".
5731.22Slukem
5741.126Swiz     syspkgs       Perform "make syspkgs".
5751.54Sapb
5761.126Swiz     iso-image     Perform "make iso-image".
5771.56Sapb
5781.105Swiz     iso-image-source
5791.126Swiz                   Perform "make iso-image-source".
5801.65Sjnemeth
5811.105Swiz     install-image
5821.126Swiz                   Perform "make install-image".
5831.97Stsutsui
5841.126Swiz     live-image    Perform "make live-image".
5851.97Stsutsui
5861.148Slukem     list-arch     Show a list of valid MACHINE and MACHINE_ARCH settings, the
5871.148Slukem                   default MACHINE_ARCH for each MACHINE, and aliases for
5881.109Sapb                   MACHINE/MACHINE_ARCH pairs, and then exits.  The -m or -a
5891.109Sapb                   options (or both) may be used to specify glob patterns that
5901.109Sapb                   will be used to narrow the list of results; for example,
5911.131Sleot                   "build.sh -m 'evb*' -a '*arm*' list-arch" will list all
5921.109Sapb                   known MACHINE/MACHINE_ARCH values in which either MACHINE
5931.109Sapb                   or ALIAS matches the pattern `evb*', and MACHINE_ARCH
5941.109Sapb                   matches the pattern `*arm*'.
5951.109Sapb
5961.105Swiz     The following command line options alter the behaviour of the build.sh
5971.51Swiz     operations described above:
5981.1Stv
5991.109Sapb     -a arch   Set the value of MACHINE_ARCH to arch.  See the -m option for
6001.109Sapb               more information.
6011.9Sthorpej
6021.105Swiz     -B buildid
6031.105Swiz               Set the value of BUILDID to buildid.  This will also append the
6041.126Swiz               build identifier to the name of the "make" wrapper script so
6051.37Slukem               that the resulting name is of the form
6061.126Swiz               "nbmake-MACHINE-BUILDID".
6071.1Stv
6081.105Swiz     -C cdextras
6091.105Swiz               Append cdextras to the CDEXTRA variable, which is a space-
6101.104Swiz               separated list of files or directories that will be added to
6111.126Swiz               the CD-ROM image that may be create by the "iso-image" or
6121.126Swiz               "iso-image-source" operations.  Files will be added to the root
6131.126Swiz               of the CD-ROM image, whereas directories will be copied
6141.87Sapb               recursively.  If relative paths are specified, they will be
6151.87Sapb               converted to absolute paths before being used.  Multiple paths
6161.105Swiz               may be specified via multiple -C options, or via a single
6171.87Sapb               option whose argument contains multiple space-separated paths.
6181.66Sjnemeth
6191.136Suwe     -c compiler
6201.136Suwe               Select the compiler for the toolchain to build NetBSD and for
6211.136Suwe               inclusion in the NetBSD distribution.  Supported choices:
6221.136Suwe
6231.136Suwe                     clang
6241.136Suwe
6251.136Suwe                     gcc [default]
6261.136Suwe
6271.136Suwe               The compiler used to build the toolchain can be different; see
6281.136Suwe               HOST_CC and HOST_CXX.
6291.136Suwe
6301.105Swiz     -D dest   Set the value of DESTDIR to dest.  If a relative path is
6311.104Swiz               specified, it will be converted to an absolute path before
6321.104Swiz               being used.
6331.104Swiz
6341.105Swiz     -E        Set `expert' mode.  This overrides various sanity checks, and
6351.105Swiz               allows: DESTDIR does not have to be set to a non-root path for
6361.105Swiz               builds, and MKUNPRIVED=yes does not have to be set when
6371.104Swiz               building as a non-root user.
6381.23Slukem
6391.105Swiz               Note: It is highly recommended that you know what you are doing
6401.23Slukem               when you use this option.
6411.19Slukem
6421.148Slukem     -h        Show a help message, and exit.
6431.51Swiz
6441.105Swiz     -j njob   Run up to njob make(1) subjobs in parallel; passed through to
6451.74Sapb               make(1).  If you see failures for reasons other than running
6461.105Swiz               out of memory while using build.sh with -j, please save
6471.104Swiz               complete build logs so the failures can be analyzed.
6481.74Sapb
6491.105Swiz               To achieve the fastest builds, -j values between (1 + the
6501.104Swiz               number of CPUs) and (2 * the number of CPUs) are recommended.
6511.104Swiz               Use lower values on machines with limited memory or I/O
6521.104Swiz               bandwidth.
6531.104Swiz
6541.126Swiz     -M obj    Set MAKEOBJDIRPREFIX to obj.  Unsets MAKEOBJDIR.  See "-O obj"
6551.126Swiz               for more information.
6561.104Swiz
6571.105Swiz               For instance, if the source directory is /usr/src, a setting of
6581.126Swiz               "-M /usr/obj" will place build-time files under
6591.105Swiz               /usr/obj/usr/src/bin, /usr/obj/usr/src/lib,
6601.105Swiz               /usr/obj/usr/src/usr.bin, and so forth.
6611.88Sapb
6621.73Sapb               If a relative path is specified, it will be converted to an
6631.105Swiz               absolute path before being used.  build.sh imposes the
6641.105Swiz               restriction that the argument to the -M option must not begin
6651.126Swiz               with a "$" (dollar sign) character; otherwise it would be too
6661.104Swiz               difficult to determine whether the value is an absolute or a
6671.104Swiz               relative path.  If the directory does not already exist,
6681.105Swiz               build.sh will create it.
6691.18Slukem
6701.109Sapb     -m mach   Set the value of MACHINE to mach, unless the mach argument is
6711.109Sapb               an alias that refers to a MACHINE/MACHINE_ARCH pair, in which
6721.109Sapb               case both MACHINE and MACHINE_ARCH are set from the alias.
6731.109Sapb               Such aliases are interpreted entirely by build.sh; they are not
6741.109Sapb               used by any other part of the build system.  The MACHINE_ARCH
6751.109Sapb               setting implied by mach will override any value of MACHINE_ARCH
6761.109Sapb               in the process environment, but will not override a value set
6771.109Sapb               by the -a option.  All cross builds require -m, but if unset on
6781.109Sapb               a NetBSD host, the host's value of MACHINE will be detected and
6791.109Sapb               used automatically.
6801.109Sapb
6811.109Sapb               See the list-arch operation for a way to get a list of valid
6821.109Sapb               MACHINE and MACHINE_ARCH settings.
6831.1Stv
6841.105Swiz     -N noiselevel
6851.126Swiz               Set the "noisyness" level of the build, by setting MAKEVERBOSE
6861.126Swiz               to noiselevel.
6871.45Slukem
6881.105Swiz     -n        Show the commands that would be executed by build.sh, but do
6891.126Swiz               not make any changes.  This is similar in concept to "make -n".
6901.1Stv
6911.105Swiz     -O obj    Create an appropriate transform macro for MAKEOBJDIR that will
6921.105Swiz               place the built object files under obj.  Unsets
6931.105Swiz               MAKEOBJDIRPREFIX.
6941.88Sapb
6951.126Swiz               For instance, a setting of "-O /usr/obj" will place build-time
6961.126Swiz               files under /usr/obj/bin, /usr/obj/lib, /usr/obj/usr.bin, and
6971.126Swiz               so forth.
6981.88Sapb
6991.88Sapb               If a relative path is specified, it will be converted to an
7001.105Swiz               absolute path before being used.  build.sh imposes the
7011.105Swiz               restriction that the argument to the -O option must not contain
7021.126Swiz               a "$" (dollar sign) character.  If the directory does not
7031.105Swiz               already exist, build.sh will create it.
7041.73Sapb
7051.105Swiz               In normal use, exactly one of the -M or -O options should be
7061.105Swiz               specified.  If neither -M nor -O is specified, then a default
7071.78Sapb               object directory will be chosen according to rules in
7081.78Sapb               <bsd.obj.mk>.  Relying on this default is not recommended
7091.78Sapb               because it is determined by complex rules that are influenced
7101.78Sapb               by the values of several variables and by the location of the
7111.78Sapb               source directory.
7121.18Slukem
7131.151Slukem               Note: Placing the obj directory location outside of the default
7141.151Slukem               source tree hierarchy makes it easier to manually clear out old
7151.151Slukem               files in the event the "make cleandir" operation is unable to
7161.151Slukem               do so.  (See CAVEATS below.)
7171.151Slukem
7181.151Slukem               Note: The use of one of -M or -O is the only means of building
7191.151Slukem               multiple machine architecture userlands from the same source
7201.151Slukem               tree without cleaning between builds (in which case, one would
7211.151Slukem               specify distinct obj locations for each).
7221.109Sapb
7231.126Swiz     -o        Set the value of MKOBJDIRS to "no".  Otherwise, it will be
7241.126Swiz               automatically set to "yes".  This default is opposite to the
7251.105Swiz               behaviour when not using build.sh.
7261.1Stv
7271.148Slukem     -P        Set the value of MKREPRO and MKREPRO_TIMESTAMP to the latest
7281.148Slukem               source CVS timestamp for reproducible builds.
7291.148Slukem
7301.105Swiz     -R rel    Set the value of RELEASEDIR to rel.  If a relative path is
7311.59Sapb               specified, it will be converted to an absolute path before
7321.59Sapb               being used.
7331.18Slukem
7341.105Swiz     -r        Remove the contents of DESTDIR and TOOLDIR before building
7351.1Stv               (provides a clean starting point).  This will skip deleting
7361.105Swiz               DESTDIR if building on a native system to the root directory.
7371.1Stv
7381.105Swiz     -S seed   Change the value of BUILDSEED to seed.  This should rarely be
7391.72Sperry               necessary.
7401.72Sperry
7411.105Swiz     -T tools  Set the value of TOOLDIR to tools.  If a relative path is
7421.104Swiz               specified, it will be converted to an absolute path before
7431.126Swiz               being used.  If set, the bootstrap "make" will only be rebuilt
7441.126Swiz               if the source files for make(1) have changed.
7451.104Swiz
7461.105Swiz     -U        Set MKUNPRIVED=yes.
7471.104Swiz
7481.105Swiz     -u        Set MKUPDATE=yes.
7491.104Swiz
7501.105Swiz     -V var=[value]
7511.105Swiz               Set the environment variable var to an optional value.  This is
7521.105Swiz               propagated to the nbmake wrapper.
7531.105Swiz
7541.105Swiz     -w wrapper
7551.105Swiz               Create the nbmake wrapper script (see below) in a custom
7561.105Swiz               location, specified by wrapper.  This allows, for instance, to
7571.151Slukem               place the wrapper in PATH automatically.
7581.151Slukem
7591.151Slukem               Note: wrapper is the full name of the file, not just a
7601.151Slukem               directory name.  If a relative path is specified, it will be
7611.151Slukem               converted to an absolute path before being used.
7621.1Stv
7631.105Swiz     -X x11src
7641.105Swiz               Set the value of X11SRCDIR to x11src.  If a relative path is
7651.59Sapb               specified, it will be converted to an absolute path before
7661.59Sapb               being used.
7671.49Slukem
7681.105Swiz     -x        Set MKX11=yes.
7691.49Slukem
7701.105Swiz     -Z var    Unset ("zap") the environment variable var.  This is propagated
7711.105Swiz               to the nbmake wrapper.
7721.40Slukem
7731.148Slukem     -?        Show a help message, and exit.
7741.148Slukem
7751.105Swiz   The "nbmake-MACHINE" wrapper script
7761.105Swiz     If using the build.sh script to build NetBSD, a nbmake-MACHINE script
7771.105Swiz     will be created in TOOLDIR/bin upon the first build to assist in building
7781.1Stv     subtrees on a cross-compile host.
7791.1Stv
7801.105Swiz     nbmake-MACHINE can be invoked in lieu of make(1), and will instead call
7811.126Swiz     the up-to-date version of "nbmake" installed into TOOLDIR/bin with
7821.105Swiz     several key variables pre-set, including MACHINE, MACHINE_ARCH, and
7831.105Swiz     TOOLDIR.  nbmake-MACHINE will also set variables specified with -V, and
7841.105Swiz     unset variables specified with -Z.
7851.40Slukem
7861.105Swiz     This script can be symlinked into a directory listed in PATH, or called
7871.1Stv     with an absolute path.
7881.1Stv
7891.105SwizEXAMPLES
7901.151Slukem     1.   % ./build.sh [OPTIONS] tools kernel=GENERIC
7911.28Slukem
7921.28Slukem          Build a new toolchain, and use the new toolchain to configure and
7931.28Slukem          build a new GENERIC kernel.
7941.28Slukem
7951.151Slukem     2.   % ./build.sh [OPTIONS] -U distribution
7961.28Slukem
7971.105Swiz          Using unprivileged mode, build a complete distribution to a DESTDIR
7981.148Slukem          directory that build.sh selects (and will show).
7991.28Slukem
8001.151Slukem     3.   # ./build.sh [OPTIONS] -U install=/
8011.28Slukem
8021.105Swiz          As root, install to / the distribution that was built by example 2.
8031.105Swiz          Even though this is run as root, -U is required so that the
8041.105Swiz          permissions stored in DESTDIR/METALOG are correctly applied to the
8051.105Swiz          files as they're copied to /.
8061.28Slukem
8071.151Slukem     4.   % ./build.sh [OPTIONS] -U -u release
8081.28Slukem
8091.105Swiz          Using unprivileged mode, build a complete release to DESTDIR and
8101.148Slukem          RELEASEDIR directories that build.sh selects (and will show).
8111.126Swiz          MKUPDATE=yes (-u) is set to prevent the "make cleandir", so that if
8121.126Swiz          this is run after example 2, it doesn't need to redo that portion of
8131.126Swiz          the release build.
8141.16Slukem
8151.105SwizSEE ALSO
8161.155Slukem     config(1), ctags(1), ex(1), gzip(1), lint(1), make(1), mandoc(1),
8171.155Slukem     mkisofs(1), nroff(1), vi(1), mk.conf(5), atf(7), hier(7), mdoc(7),
8181.155Slukem     release(7), etcupdate(8), installboot(8), mount(8), postinstall(8),
8191.151Slukem     sysinst(8), pkgsrc/sysutils/cdrtools
8201.1Stv
8211.155Slukem     Note: The NetBSD manual pages are also available at
8221.155Slukem     https://man.netbsd.org
8231.155Slukem
8241.105SwizHISTORY
8251.105Swiz     The build.sh based build scheme was introduced for NetBSD 1.6 as
8261.105Swiz     USE_NEW_TOOLCHAIN, and re-worked to TOOLCHAIN_MISSING after that.
8271.1Stv
8281.109SapbCAVEATS
8291.109Sapb     After significant updates to third-party components in the source tree,
8301.126Swiz     the "make cleandir" operation may be insufficient to clean out old files
8311.126Swiz     in object directories.  Instead, one may have to manually remove the
8321.126Swiz     files.  Consult the UPDATING file for notices concerning this.
8331.109Sapb
8341.157SlukemNetBSD                           July 18, 2023                          NetBSD
835