BUILDING revision 1.159
11.104SwizBUILDING(8)                 System Manager's Manual                BUILDING(8)
21.1Stv
31.105SwizNAME
41.150Slukem     BUILDING - Procedure for building NetBSD from source code
51.1Stv
61.105SwizREQUIREMENTS
71.1Stv     NetBSD is designed to be buildable on most POSIX-compliant host systems.
81.105Swiz     The basic build procedure is the same whether compiling natively (on the
91.105Swiz     same NetBSD architecture) or cross compiling (on another architecture or
101.1Stv     OS).
111.1Stv
121.157Slukem     This source tree contains the build.sh shell script which supports both
131.157Slukem     native and cross builds of NetBSD.
141.157Slukem
151.126Swiz     This source tree contains a special subtree, "tools", which uses the host
161.126Swiz     system to create a build toolchain for the target architecture.  The host
171.126Swiz     system must have at least C and C++ compilers in order to create the
181.157Slukem     toolchain (make(1) is not required); all other tools (including make(1)
191.157Slukem     as nbmake) are created as part of the NetBSD build process.  (See the
201.157Slukem     Environment variables section below if you need to override or manually
211.157Slukem     select your compilers.)
221.1Stv
231.155Slukem     Note: Within this document, cross-references to manual pages are to the
241.155Slukem     NetBSD manual pages, not the host system manual pages.  The mdoc(7)
251.155Slukem     source to the NetBSD manual pages can be found within the source tree,
261.155Slukem     and these and can be formatted with mandoc(1) or nroff(1) if those are
271.155Slukem     available on the host system.  The NetBSD manual pages are also available
281.155Slukem     at https://man.netbsd.org
291.155Slukem
301.105SwizFILES
311.105Swiz   Source tree layout
321.151Slukem     BUILDING       This document (in plaintext).  Generated from
331.151Slukem                    doc/BUILDING.mdoc.
341.48Sjmc
351.105Swiz     Makefile       The main Makefile for NetBSD; should only be run for
361.44Slukem                    native builds with an appropriately up-to-date version of
371.142Sandvar                    NetBSD make(1).  Intended for expert use with knowledge of
381.129Ssevan                    its shortcomings, it has been superseded by the build.sh
391.129Ssevan                    shell script as the recommended means for building NetBSD.
401.1Stv
411.105Swiz     UPDATING       Special notes for updating from an earlier revision of
421.1Stv                    NetBSD.  It is important to read this file before every
431.1Stv                    build of an updated source tree.
441.1Stv
451.105Swiz     build.sh       Bourne-compatible shell script used for building the host
461.1Stv                    build tools and the NetBSD system from scratch.  Can be
471.1Stv                    used for both native and cross builds, and should be used
481.129Ssevan                    instead of make(1) as it performs additional checks to
491.129Ssevan                    prevent common issues going undetected, such as building
501.129Ssevan                    with an outdated version of make(1).
511.1Stv
521.105Swiz     crypto/dist/, dist/, gnu/dist/
531.104Swiz                    Sources imported verbatim from third parties, without
541.104Swiz                    mangling the existing build structure.  Other source trees
551.126Swiz                    in bin through usr.sbin use the NetBSD make(1) "reachover"
561.126Swiz                    Makefile semantics when building these programs for a
571.126Swiz                    native host.
581.104Swiz
591.151Slukem     distrib/, etc/
601.151Slukem                    Sources for items used when making a full release
611.151Slukem                    snapshot, such as files installed in DESTDIR/etc on the
621.151Slukem                    destination system, boot media, and release notes.
631.151Slukem
641.151Slukem     doc/BUILDING.mdoc
651.155Slukem                    The source to this document, in mdoc(7) format.  Used to
661.155Slukem                    generate BUILDING.
671.151Slukem
681.158Slukem     external/, sys/external/
691.106Swiz                    Sources and build infrastructure for components imported
701.106Swiz                    (mostly) unchanged from upstream maintainers, sorted by
711.106Swiz                    applicable license.  This is (slowly) replacing the
721.106Swiz                    crypto/dist, dist, and gnu/dist directories.
731.106Swiz
741.151Slukem     external/mit/xorg/
751.151Slukem                    "Reachover" build structure for modular Xorg; the source
761.151Slukem                    is in X11SRCDIR.
771.1Stv
781.157Slukem     mk.conf        Optional source tree specific mk.conf(5), used (if
791.157Slukem                    present) instead of /etc/mk.conf unless MAKECONF is
801.157Slukem                    defined.
811.157Slukem
821.157Slukem                    Note: Not part of the NetBSD source repository.
831.157Slukem
841.151Slukem     regress/, tests/
851.70Sdholland                    Regression test harness.  Can be cross-compiled, but only
861.105Swiz                    run natively.  tests/ uses the atf(7) test framework;
871.105Swiz                    regress/ contains older tests that have not yet been
881.70Sdholland                    migrated to atf(7).
891.1Stv
901.105Swiz     sys/           NetBSD kernel sources.
911.1Stv
921.126Swiz     tools/         "Reachover" build structure for the host build tools.
931.104Swiz                    This has a special method of determining out-of-date
941.104Swiz                    status.
951.1Stv
961.151Slukem     tools/compat/README
971.151Slukem                    Special notes for cross-hosting a NetBSD build on non-
981.151Slukem                    NetBSD platforms.
991.151Slukem
1001.151Slukem     Other directories including bin/ ... usr.sbin/
1011.1Stv                    Sources to the NetBSD userland (non-kernel) programs.  If
1021.1Stv                    any of these directories are missing, they will be skipped
1031.1Stv                    during the build.
1041.1Stv
1051.105Swiz   Build tree layout
1061.156Slukem     The NetBSD build tree is described in hier(7) (whose mdoc(7) source is in
1071.155Slukem     share/man/man7/hier.7), and the release layout is described in release(7)
1081.156Slukem     (whose mdoc(7) source is in share/man/man7/release.7).
1091.1Stv
1101.105SwizCONFIGURATION
1111.105Swiz   Environment variables
1121.8Slukem     Several environment variables control the behaviour of NetBSD builds.
1131.8Slukem
1141.152Slukem     HOST_CC         Path name to C compiler used to create the toolchain.
1151.58Sapb
1161.155Slukem                     Default: "cc".
1171.155Slukem
1181.152Slukem     HOST_CFLAGS     Flags passed to the host C compiler.
1191.53Sreed
1201.155Slukem                     Default: "-O".
1211.155Slukem
1221.155Slukem     HOST_CPPFLAGS   Flags passed to the host C/C++ pre-processor.
1231.155Slukem
1241.155Slukem                     Default: Unset.
1251.155Slukem
1261.152Slukem     HOST_CXX        Path name to C++ compiler used to create the toolchain.
1271.137Suwe
1281.155Slukem                     Default: Unset, but defaults to "c++" where required.
1291.155Slukem
1301.152Slukem     HOST_CXXFLAGS   Flags passed to the host C++ compiler.
1311.53Sreed
1321.155Slukem                     Default: Unset.
1331.155Slukem
1341.152Slukem     HOST_SH         Path name to a shell available on the host system and
1351.152Slukem                     suitable for use during the build.  The NetBSD build
1361.152Slukem                     system requires a modern Bourne-like shell with POSIX-
1371.152Slukem                     compliant features, and also requires support for the
1381.152Slukem                     "local" keyword to declare local variables in shell
1391.152Slukem                     functions (which is a widely-implemented but non-
1401.152Slukem                     standardised feature).
1411.152Slukem
1421.152Slukem                     Depending on the host system, a suitable shell may be
1431.152Slukem                     /bin/sh, /usr/xpg4/bin/sh, /bin/ksh (provided it is a
1441.152Slukem                     variant of ksh that supports the "local" keyword, such as
1451.152Slukem                     ksh88, but not ksh93), or /usr/local/bin/bash.
1461.152Slukem
1471.152Slukem                     Most parts of the build require HOST_SH to be an absolute
1481.152Slukem                     path; however, build.sh allows it to be a simple command
1491.152Slukem                     name, which will be converted to an absolute path by
1501.152Slukem                     searching the PATH.
1511.137Suwe
1521.155Slukem                     Default: "sh".
1531.155Slukem
1541.147Slukem     INSTALLBOOT_UBOOT_PATHS
1551.152Slukem                     A colon-separated list of search paths used by
1561.152Slukem                     installboot(8) to find U-Boot packages.
1571.147Slukem
1581.155Slukem                     Default: Unset.
1591.155Slukem
1601.152Slukem     MACHINE         Machine type, e.g., "macppc".
1611.8Slukem
1621.155Slukem                     Default: Unset.
1631.155Slukem
1641.152Slukem     MACHINE_ARCH    Machine architecture, e.g., "powerpc".
1651.8Slukem
1661.155Slukem                     Default: Unset.
1671.155Slukem
1681.152Slukem     MAKE            Path name to invoke make(1) as.
1691.8Slukem
1701.155Slukem                     Default: "make".
1711.155Slukem
1721.152Slukem     MAKECONF        The name of the make(1) configuration file.  See "make"
1731.152Slukem                     variables and mk.conf(5).
1741.151Slukem
1751.152Slukem                     Note: Only settable in the process environment.
1761.151Slukem
1771.157Slukem                     Default: "/etc/mk.conf", although build.sh will set the
1781.157Slukem                     default to the full path to mk.conf if the latter is
1791.157Slukem                     present in the same directory as build.sh.
1801.151Slukem
1811.152Slukem     MAKEFLAGS       Flags to invoke make(1) with.
1821.151Slukem
1831.152Slukem                     Note: build.sh ignores the value of MAKEFLAGS passed in
1841.152Slukem                     the environment, but allows MAKEFLAGS to be set via the
1851.152Slukem                     -V option.
1861.151Slukem
1871.155Slukem                     Default: "-X" on systems with a small ARG_MAX (Cygwin,
1881.155Slukem                     Darwin, FreeBSD); otherwise unset.
1891.155Slukem
1901.152Slukem     MAKEOBJDIR      Directory to use as the .OBJDIR for the current
1911.152Slukem                     directory.  The value is subjected to variable expansion
1921.152Slukem                     by make(1).  Typical usage is to set this variable to a
1931.152Slukem                     value involving the use of `${.CURDIR:S...}' or
1941.152Slukem                     `${.CURDIR:C...}', to derive the value of .OBJDIR from
1951.152Slukem                     the value of .CURDIR.  Used only if MAKEOBJDIRPREFIX is
1961.152Slukem                     not defined.
1971.151Slukem
1981.152Slukem                     Note: MAKEOBJDIR can be provided only in the environment
1991.152Slukem                     or via the -O flag of build.sh; it cannot usefully be set
2001.152Slukem                     inside a Makefile, including in mk.conf(5) or MAKECONF.
2011.151Slukem
2021.155Slukem                     Default: Unset.
2031.155Slukem
2041.151Slukem     MAKEOBJDIRPREFIX
2051.152Slukem                     Top level directory of the object directory tree.  The
2061.152Slukem                     value is subjected to variable expansion by make(1).
2071.152Slukem                     build.sh will create the ${MAKEOBJDIRPREFIX} directory if
2081.152Slukem                     necessary, but if make(1) is used without build.sh, then
2091.152Slukem                     rules in <bsd.obj.mk> will abort the build if the
2101.152Slukem                     ${MAKEOBJDIRPREFIX} directory does not exist.  If the
2111.152Slukem                     value is defined and valid, then
2121.152Slukem                     ${MAKEOBJDIRPREFIX}/${.CURDIR} is used as the .OBJDIR for
2131.152Slukem                     the current directory.  The current directory may be read
2141.152Slukem                     only.
2151.152Slukem
2161.152Slukem                     Note: MAKEOBJDIRPREFIX can be provided only in the
2171.152Slukem                     environment or via the -M flag of build.sh; it cannot
2181.152Slukem                     usefully be set inside a Makefile, including in
2191.152Slukem                     mk.conf(5) or MAKECONF.
2201.152Slukem
2211.155Slukem                     Default: Unset.
2221.155Slukem
2231.152Slukem     TMPDIR          Top-level directory to store temporary directories used
2241.152Slukem                     by build.sh before paths to other directories such as
2251.152Slukem                     .OBJDIR can be determined.
2261.149Slukem
2271.152Slukem                     Note: Must support execution of binaries.  I.e., without
2281.152Slukem                     mount(8)'s -o noexec option.
2291.149Slukem
2301.152Slukem                     Default: "/tmp".
2311.149Slukem
2321.105Swiz   "make" variables
2331.156Slukem     Variables that control the behavior of NetBSD builds are documented in
2341.156Slukem     mk.conf(5) (whose mdoc(7) source is in share/man/man5/mk.conf.5).
2351.45Slukem
2361.155Slukem     Unless otherwise specified, these variables may be set in either the
2371.155Slukem     process environment or the make(1) configuration file mk.conf(5)
2381.155Slukem     specified by MAKECONF.
2391.49Slukem
2401.159Slukem     Note: Variables set in the environment, either directly or via build.sh
2411.159Slukem     options to set specific values in the nbmake-MACHINE wrapper script do
2421.159Slukem     not override variables set in the mk.conf(5) file.  To allow variables in
2431.159Slukem     mk.conf(5) to be overridden by the environment or build.sh options,
2441.159Slukem     define the variables using the "?=" make(1) variable assignment operator.
2451.159Slukem     For example,
2461.159Slukem
2471.159Slukem           MAKEVERBOSE?=1
2481.159Slukem
2491.105SwizBUILDING
2501.105Swiz   "make" command line options
2511.52Sjmc     This is not a summary of all the options available to make(1); only the
2521.52Sjmc     options used most frequently with NetBSD builds are listed here.
2531.1Stv
2541.105Swiz     -j njob    Run up to njob make(1) subjobs in parallel.  Makefiles should
2551.52Sjmc                use .WAIT or have explicit dependencies as necessary to
2561.74Sapb                enforce build ordering.
2571.10Ssommerfe
2581.105Swiz     -m dir     Specify the default directory for searching for system
2591.104Swiz                Makefile segments, mainly the <bsd.*.mk> files.  When building
2601.104Swiz                any full NetBSD source tree, this should be set to the
2611.126Swiz                "share/mk" directory in the source tree.  This is set
2621.104Swiz                automatically when building from the top level, or when using
2631.105Swiz                build.sh.
2641.1Stv
2651.148Slukem     -n         Show the commands that would have been executed, but do not
2661.1Stv                actually execute them.  This will still cause recursion to
2671.1Stv                take place.
2681.1Stv
2691.148Slukem     -V var     Show make(1)'s idea of the value of var.  Does not build any
2701.1Stv                targets.
2711.1Stv
2721.105Swiz     var=value  Set the variable var to value, overriding any setting
2731.105Swiz                specified by the process environment, the MAKECONF
2741.104Swiz                configuration file, or the system Makefile segments.
2751.1Stv
2761.105Swiz   "make" targets
2771.1Stv     These default targets may be built by running make(1) in any subtree of
2781.1Stv     the NetBSD source code.  It is recommended that none of these be used
2791.126Swiz     from the top level Makefile; as a specific exception, "make obj" and
2801.126Swiz     "make cleandir" are useful in that context.
2811.1Stv
2821.105Swiz     all        Build programs, libraries, and preformatted documentation.
2831.1Stv
2841.105Swiz     clean      Remove program and library object code files.
2851.1Stv
2861.105Swiz     cleandir   Same as clean, but also remove preformatted documentation,
2871.126Swiz                dependency files generated by "make depend", and any other
2881.49Slukem                files known to be created at build time.
2891.1Stv
2901.105Swiz     depend     Create dependency files (.depend) containing more detailed
2911.44Slukem                information about the dependencies of source code on header
2921.1Stv                files.  Allows programs to be recompiled automatically when a
2931.1Stv                dependency changes.
2941.1Stv
2951.126Swiz     dependall  Does a "make depend" immediately followed by a "make all".
2961.10Ssommerfe                This improves cache locality of the build since both passes
2971.10Ssommerfe                read the source files in their entirety.
2981.1Stv
2991.105Swiz     distclean  Synonym for cleandir.
3001.49Slukem
3011.105Swiz     includes   Build and install system header files.  Typically needed
3021.44Slukem                before any system libraries or programs can be built.
3031.1Stv
3041.105Swiz     install    Install programs, libraries, and documentation into DESTDIR.
3051.105Swiz                Few files will be installed to DESTDIR/dev, DESTDIR/etc,
3061.105Swiz                DESTDIR/root or DESTDIR/var in order to prevent user supplied
3071.27Slukem                configuration data from being overwritten.
3081.1Stv
3091.105Swiz     lint       Run lint(1) against the C source code, where appropriate, and
3101.1Stv                generate system-installed lint libraries.
3111.1Stv
3121.105Swiz     obj        Create object directories to be used for built files, instead
3131.1Stv                of building directly in the source tree.
3141.1Stv
3151.105Swiz     tags       Create ctags(1) searchable function lists usable by the ex(1)
3161.1Stv                and vi(1) text editors.
3171.1Stv
3181.105Swiz   "make" targets for the top level
3191.1Stv     Additional make(1) targets are usable specifically from the top source
3201.1Stv     level to facilitate building the entire NetBSD source tree.
3211.1Stv
3221.105Swiz     build         Build the entire NetBSD system (except the kernel).  This
3231.68Sreed                   orders portions of the source tree such that prerequisites
3241.68Sreed                   will be built in the proper order.
3251.20Slukem
3261.126Swiz     distribution  Do a "make build", and then install a full distribution
3271.105Swiz                   (which does not include a kernel) into DESTDIR, including
3281.105Swiz                   files in DESTDIR/dev, DESTDIR/etc, DESTDIR/root and
3291.105Swiz                   DESTDIR/var.
3301.20Slukem
3311.126Swiz     buildworld    As per "make distribution", except that it ensures that
3321.105Swiz                   DESTDIR is not the root directory.
3331.20Slukem
3341.105Swiz     installworld  Install the distribution from DESTDIR to INSTALLWORLDDIR,
3351.70Sdholland                   which defaults to the root directory.  Ensures that
3361.105Swiz                   INSTALLWORLDDIR is not the root directory if cross
3371.104Swiz                   compiling.
3381.20Slukem
3391.105Swiz                   The INSTALLSETS environment variable may be set to a space-
3401.95Sapb                   separated list of distribution sets to be installed.  By
3411.126Swiz                   default, all sets except "etc" and "xetc" are installed, so
3421.126Swiz                   most files in INSTALLWORLDDIR/etc will not be installed or
3431.126Swiz                   modified.
3441.61Sapb
3451.105Swiz                   Note: Before performing this operation with
3461.105Swiz                   INSTALLWORLDDIR=/, it is highly recommended that you
3471.61Sapb                   upgrade your kernel and reboot.  After performing this
3481.61Sapb                   operation, it is recommended that you use etcupdate(8) to
3491.105Swiz                   update files in INSTALLWORLDDIR/etc, and postinstall(8) to
3501.102Sabs                   check for or fix inconsistencies.
3511.20Slukem
3521.105Swiz     sets          Create distribution sets from DESTDIR into
3531.105Swiz                   RELEASEDIR/RELEASEMACHINEDIR/binary/sets.  Should be run
3541.126Swiz                   after "make distribution", as "make build" alone does not
3551.126Swiz                   install all of the required files.
3561.25Slukem
3571.105Swiz     sourcesets    Create source sets of the source tree into
3581.105Swiz                   RELEASEDIR/source/sets.
3591.31Slukem
3601.105Swiz     syspkgs       Create syspkgs from DESTDIR into
3611.105Swiz                   RELEASEDIR/RELEASEMACHINEDIR/binary/syspkgs.  Should be run
3621.126Swiz                   after "make distribution", as "make build" alone does not
3631.126Swiz                   install all of the required files.
3641.54Sapb
3651.126Swiz     release       Do a "make distribution", build kernels, distribution
3661.126Swiz                   media, and install sets (this as per "make sets"), and then
3671.126Swiz                   package the system into a standard release layout as
3681.105Swiz                   described by release(7).  This requires that RELEASEDIR be
3691.25Slukem                   set (see above).
3701.1Stv
3711.105Swiz     iso-image     Create a NetBSD installation CD-ROM image in the
3721.114Ssnj                   RELEASEDIR/images directory.  The CD-ROM file system will
3731.114Ssnj                   have a layout as described in release(7).
3741.62Sapb
3751.62Sapb                   For most machine types, the CD-ROM will be bootable, and
3761.104Swiz                   will automatically run the sysinst(8) menu-based
3771.104Swiz                   installation program, which can be used to install or
3781.104Swiz                   upgrade a NetBSD system.  Bootable CD-ROMs also contain
3791.104Swiz                   tools that may be useful in repairing a damaged NetBSD
3801.104Swiz                   installation.
3811.62Sapb
3821.126Swiz                   Before "make iso-image" is attempted, RELEASEDIR must be
3831.126Swiz                   populated by "make release" or equivalent.
3841.62Sapb
3851.151Slukem                   Note: Other, smaller, CD-ROM images may be created in the
3861.151Slukem                   RELEASEDIR/RELEASEMACHINEDIR/installation/cdrom directory
3871.151Slukem                   by "make release".  These smaller images usually contain
3881.151Slukem                   the same tools as the larger images in RELEASEDIR/images,
3891.151Slukem                   but do not contain additional content such as the
3901.151Slukem                   distribution sets.
3911.62Sapb
3921.151Slukem                   Note: The mac68k port still uses an older method of
3931.104Swiz                   creating CD-ROM images.  This requires the mkisofs(1)
3941.104Swiz                   utility, which is not part of NetBSD, but which can be
3951.105Swiz                   installed from pkgsrc/sysutils/cdrtools.
3961.56Sapb
3971.105Swiz     iso-image-source
3981.64Sjnemeth                   Create a NetBSD installation CD-ROM image in the
3991.114Ssnj                   RELEASEDIR/images directory.  The CD-ROM file system will
4001.114Ssnj                   have a layout as described in release(7).  It will have top
4011.64Sjnemeth                   level directories for the machine type and source.
4021.64Sjnemeth
4031.64Sjnemeth                   For most machine types, the CD-ROM will be bootable, and
4041.104Swiz                   will automatically run the sysinst(8) menu-based
4051.104Swiz                   installation program, which can be used to install or
4061.104Swiz                   upgrade a NetBSD system.  Bootable CD-ROMs also contain
4071.104Swiz                   tools that may be useful in repairing a damaged NetBSD
4081.104Swiz                   installation.
4091.64Sjnemeth
4101.126Swiz                   Before "make iso-image-source" is attempted, RELEASEDIR
4111.126Swiz                   must be populated by "make sourcesets release" or
4121.104Swiz                   equivalent.
4131.64Sjnemeth
4141.151Slukem                   Note: Other, smaller, CD-ROM images may be created in the
4151.151Slukem                   RELEASEDIR/RELEASEMACHINEDIR/installation/cdrom directory
4161.151Slukem                   by "make release".  These smaller images usually contain
4171.151Slukem                   the same tools as the larger images in RELEASEDIR/images,
4181.151Slukem                   but do not contain additional content such as the
4191.151Slukem                   distribution sets.
4201.64Sjnemeth
4211.151Slukem                   Note: The mac68k port still uses an older method of
4221.104Swiz                   creating CD-ROM images.  This requires the mkisofs(1)
4231.104Swiz                   utility, which is not part of NetBSD, but which can be
4241.105Swiz                   installed from pkgsrc/sysutils/cdrtools.
4251.64Sjnemeth
4261.105Swiz     install-image
4271.98Sapb                   Create a bootable NetBSD installation disk image in the
4281.127Splunky                   RELEASEDIR/images directory.  The installation disk image
4291.127Splunky                   is suitable for copying to bootable USB flash memory
4301.127Splunky                   sticks, etc., for machines which are able to boot from such
4311.127Splunky                   devices.  The file system in the bootable disk image will
4321.127Splunky                   have a layout as described in release(7).
4331.97Stsutsui
4341.97Stsutsui                   The installation image is bootable, and will automatically
4351.97Stsutsui                   run the sysinst(8) menu-based installation program, which
4361.98Sapb                   can be used to install or upgrade a NetBSD system.  The
4371.98Sapb                   image also contains tools that may be useful in repairing a
4381.98Sapb                   damaged NetBSD installation.
4391.97Stsutsui
4401.126Swiz                   Before "make install-image" is attempted, RELEASEDIR must
4411.126Swiz                   be populated by "make release" or equivalent.  The build
4421.126Swiz                   must have been performed with MKUNPRIVED=yes because "make
4431.126Swiz                   install-image" relies on information in DESTDIR/METALOG.
4441.97Stsutsui
4451.120Spgoyette     live-image    Create NetBSD live images in the RELEASEDIR/images
4461.104Swiz                   directory.  The live image contains all necessary files to
4471.104Swiz                   boot NetBSD up to multi-user mode, including all files
4481.104Swiz                   which should be extracted during installation, NetBSD
4491.104Swiz                   disklabel, bootloaders, etc.
4501.104Swiz
4511.104Swiz                   The live image is suitable for use as a disk image in
4521.104Swiz                   virtual machine environments such as QEMU, and also useful
4531.104Swiz                   to boot NetBSD from a USB flash memory stick on a real
4541.98Sapb                   machine, without the need for installation.
4551.97Stsutsui
4561.126Swiz                   Before "make live-image" is attempted, RELEASEDIR must be
4571.126Swiz                   populated by "make release" or equivalent.  The build must
4581.126Swiz                   have been performed with MKUNPRIVED=yes because "make
4591.126Swiz                   install-image" relies on information in DESTDIR/METALOG.
4601.97Stsutsui
4611.105Swiz     regression-tests
4621.20Slukem                   Can only be run after building the regression tests in the
4631.126Swiz                   directory "regress".  Runs those compiled regression tests
4641.151Slukem                   on the local host.
4651.151Slukem
4661.151Slukem                   Note: Most tests are now managed instead using atf(7); this
4671.151Slukem                   target should probably run those as well but currently does
4681.151Slukem                   not.
4691.1Stv
4701.105Swiz   The "build.sh" script
4711.96Sapb     This script file is a shell script designed to build the entire NetBSD
4721.104Swiz     system on any host with a suitable modern shell and some common
4731.105Swiz     utilities.  The required shell features are described under the HOST_SH
4741.104Swiz     variable.
4751.96Sapb
4761.96Sapb     If a host system's default shell does support the required features, then
4771.96Sapb     we suggest that you explicitly specify a suitable shell using a command
4781.96Sapb     like
4791.96Sapb
4801.105Swiz           /path/to/suitable/shell build.sh [options]
4811.96Sapb
4821.105Swiz     The above command will usually enable build.sh to automatically set
4831.105Swiz     HOST_SH=/path/to/suitable/shell, but if that fails, then the following
4841.96Sapb     set of commands may be used instead:
4851.96Sapb
4861.105Swiz           HOST_SH=/path/to/suitable/shell
4871.96Sapb           export HOST_SH
4881.105Swiz           ${HOST_SH} build.sh [options]
4891.96Sapb
4901.105Swiz     If build.sh detects that it is being executed under an unsuitable shell,
4911.148Slukem     it attempts to exec a suitable shell instead, or shows an error message.
4921.105Swiz     If HOST_SH is not set explicitly, then build.sh sets a default using
4931.96Sapb     heuristics dependent on the host platform, or from the shell under which
4941.105Swiz     build.sh is executed (if that can be determined), or using the first copy
4951.105Swiz     of sh found in PATH.
4961.1Stv
4971.1Stv     All cross-compile builds, and most native builds, of the entire system
4981.126Swiz     should make use of build.sh rather than just running "make".  This way,
4991.104Swiz     the make(1) program will be bootstrapped properly, in case the host
5001.126Swiz     system has an older or incompatible "make" program.
5011.1Stv
5021.105Swiz     When compiling the entire system via build.sh, many make(1) variables are
5031.1Stv     set for you in order to help encapsulate the build process.  In the list
5041.105Swiz     of options below, variables that are automatically set by build.sh are
5051.1Stv     noted where applicable.
5061.1Stv
5071.105Swiz     The following operations are supported by build.sh:
5081.22Slukem
5091.126Swiz     build         Build the system as per "make build".  Before the main part
5101.126Swiz                   of the build commences, this command runs the obj operation
5111.126Swiz                   (unless the -o option is given), "make cleandir" (unless
5121.126Swiz                   the -u option is given), and the tools operation.
5131.22Slukem
5141.126Swiz     distribution  Build a full distribution as per "make distribution".  This
5151.126Swiz                   command first runs the build operation.
5161.22Slukem
5171.126Swiz     release       Build a full release as per "make release".  This command
5181.105Swiz                   first runs the distribution operation.
5191.22Slukem
5201.148Slukem     help          Show a help message, and exit.
5211.148Slukem
5221.159Slukem     makewrapper   Create the nbmake-MACHINE wrapper script.  This operation
5231.159Slukem                   is automatically performed for any of the other operations.
5241.23Slukem
5251.126Swiz     cleandir      Perform "make cleandir".
5261.75Slukem
5271.126Swiz     obj           Perform "make obj".
5281.22Slukem
5291.105Swiz     tools         Build and install the host tools from src/tools.  This
5301.126Swiz                   command will first run "make obj" and "make cleandir" in
5311.126Swiz                   the tools subdirectory unless the -o or -u options
5321.104Swiz                   (respectively) are given.
5331.22Slukem
5341.126Swiz     install=idir  Install the contents of DESTDIR to idir, using "make
5351.151Slukem                   installworld".
5361.151Slukem
5371.151Slukem                   Note: Files that are part of the "etc" or "xetc" sets will
5381.151Slukem                   not be installed, unless overridden by the INSTALLSETS
5391.151Slukem                   environment variable.
5401.35Slukem
5411.105Swiz     kernel=kconf  Build a new kernel.  The kconf argument is the name of a
5421.105Swiz                   configuration file suitable for use by config(1).  If kconf
5431.22Slukem                   does not contain any `/' characters, the configuration file
5441.105Swiz                   is expected to be found in the KERNCONFDIR directory, which
5451.105Swiz                   is typically sys/arch/MACHINE/conf.  The new kernel will be
5461.105Swiz                   built in a subdirectory of KERNOBJDIR, which is typically
5471.105Swiz                   sys/arch/MACHINE/compile or an associated object directory.
5481.70Sdholland
5491.105Swiz                   This command does not imply the tools command; run the
5501.105Swiz                   tools command first unless it is certain that the tools
5511.70Sdholland                   already exist and are up to date.
5521.70Sdholland
5531.126Swiz                   This command will run "make cleandir" on the kernel in
5541.105Swiz                   question first unless the -u option is given.
5551.22Slukem
5561.106Swiz     kernel.gdb=kconf
5571.106Swiz                   Build a new kernel with debug information.  Similar to the
5581.106Swiz                   above kernel=kconf operation, but creates a netbsd.gdb file
5591.106Swiz                   alongside of the kernel netbsd, which contains a full
5601.106Swiz                   symbol table and can be used for debugging (for example
5611.106Swiz                   with a cross-gdb built by MKCROSSGDB).
5621.106Swiz
5631.113Suebayasi     kernels       This command will build all kernels defined in port
5641.113Suebayasi                   specific release build procedure.
5651.111Suebayasi
5661.112Suebayasi                   This command internally calls the kernel=kconf operation
5671.112Suebayasi                   for each found kernel configuration file.
5681.111Suebayasi
5691.105Swiz     modules       This command will build kernel modules and install them
5701.105Swiz                   into DESTDIR.
5711.86Sjnemeth
5721.105Swiz     releasekernel=kconf
5731.70Sdholland                   Install a gzip(1)ed copy of the kernel previously built by
5741.105Swiz                   kernel=kconf into
5751.105Swiz                   RELEASEDIR/RELEASEMACHINEDIR/binary/kernel, usually as
5761.126Swiz                   netbsd-kconf.gz, although the "netbsd" prefix is determined
5771.126Swiz                   from the "config" directives in kconf.
5781.24Slukem
5791.126Swiz     sets          Perform "make sets".
5801.31Slukem
5811.126Swiz     sourcesets    Perform "make sourcesets".
5821.22Slukem
5831.126Swiz     syspkgs       Perform "make syspkgs".
5841.54Sapb
5851.126Swiz     iso-image     Perform "make iso-image".
5861.56Sapb
5871.105Swiz     iso-image-source
5881.126Swiz                   Perform "make iso-image-source".
5891.65Sjnemeth
5901.105Swiz     install-image
5911.126Swiz                   Perform "make install-image".
5921.97Stsutsui
5931.126Swiz     live-image    Perform "make live-image".
5941.97Stsutsui
5951.148Slukem     list-arch     Show a list of valid MACHINE and MACHINE_ARCH settings, the
5961.148Slukem                   default MACHINE_ARCH for each MACHINE, and aliases for
5971.109Sapb                   MACHINE/MACHINE_ARCH pairs, and then exits.  The -m or -a
5981.109Sapb                   options (or both) may be used to specify glob patterns that
5991.109Sapb                   will be used to narrow the list of results; for example,
6001.131Sleot                   "build.sh -m 'evb*' -a '*arm*' list-arch" will list all
6011.109Sapb                   known MACHINE/MACHINE_ARCH values in which either MACHINE
6021.109Sapb                   or ALIAS matches the pattern `evb*', and MACHINE_ARCH
6031.109Sapb                   matches the pattern `*arm*'.
6041.109Sapb
6051.105Swiz     The following command line options alter the behaviour of the build.sh
6061.51Swiz     operations described above:
6071.1Stv
6081.109Sapb     -a arch   Set the value of MACHINE_ARCH to arch.  See the -m option for
6091.109Sapb               more information.
6101.9Sthorpej
6111.105Swiz     -B buildid
6121.105Swiz               Set the value of BUILDID to buildid.  This will also append the
6131.159Slukem               build identifier to the name of the nbmake-MACHINE wrapper
6141.159Slukem               script so that the resulting name is of the form
6151.126Swiz               "nbmake-MACHINE-BUILDID".
6161.1Stv
6171.105Swiz     -C cdextras
6181.105Swiz               Append cdextras to the CDEXTRA variable, which is a space-
6191.104Swiz               separated list of files or directories that will be added to
6201.126Swiz               the CD-ROM image that may be create by the "iso-image" or
6211.126Swiz               "iso-image-source" operations.  Files will be added to the root
6221.126Swiz               of the CD-ROM image, whereas directories will be copied
6231.87Sapb               recursively.  If relative paths are specified, they will be
6241.87Sapb               converted to absolute paths before being used.  Multiple paths
6251.105Swiz               may be specified via multiple -C options, or via a single
6261.87Sapb               option whose argument contains multiple space-separated paths.
6271.66Sjnemeth
6281.136Suwe     -c compiler
6291.136Suwe               Select the compiler for the toolchain to build NetBSD and for
6301.136Suwe               inclusion in the NetBSD distribution.  Supported choices:
6311.136Suwe
6321.136Suwe                     clang
6331.136Suwe
6341.136Suwe                     gcc [default]
6351.136Suwe
6361.136Suwe               The compiler used to build the toolchain can be different; see
6371.136Suwe               HOST_CC and HOST_CXX.
6381.136Suwe
6391.105Swiz     -D dest   Set the value of DESTDIR to dest.  If a relative path is
6401.104Swiz               specified, it will be converted to an absolute path before
6411.104Swiz               being used.
6421.104Swiz
6431.105Swiz     -E        Set `expert' mode.  This overrides various sanity checks, and
6441.105Swiz               allows: DESTDIR does not have to be set to a non-root path for
6451.105Swiz               builds, and MKUNPRIVED=yes does not have to be set when
6461.104Swiz               building as a non-root user.
6471.23Slukem
6481.105Swiz               Note: It is highly recommended that you know what you are doing
6491.23Slukem               when you use this option.
6501.19Slukem
6511.148Slukem     -h        Show a help message, and exit.
6521.51Swiz
6531.105Swiz     -j njob   Run up to njob make(1) subjobs in parallel; passed through to
6541.74Sapb               make(1).  If you see failures for reasons other than running
6551.105Swiz               out of memory while using build.sh with -j, please save
6561.104Swiz               complete build logs so the failures can be analyzed.
6571.74Sapb
6581.105Swiz               To achieve the fastest builds, -j values between (1 + the
6591.104Swiz               number of CPUs) and (2 * the number of CPUs) are recommended.
6601.104Swiz               Use lower values on machines with limited memory or I/O
6611.104Swiz               bandwidth.
6621.104Swiz
6631.126Swiz     -M obj    Set MAKEOBJDIRPREFIX to obj.  Unsets MAKEOBJDIR.  See "-O obj"
6641.126Swiz               for more information.
6651.104Swiz
6661.105Swiz               For instance, if the source directory is /usr/src, a setting of
6671.126Swiz               "-M /usr/obj" will place build-time files under
6681.105Swiz               /usr/obj/usr/src/bin, /usr/obj/usr/src/lib,
6691.105Swiz               /usr/obj/usr/src/usr.bin, and so forth.
6701.88Sapb
6711.73Sapb               If a relative path is specified, it will be converted to an
6721.105Swiz               absolute path before being used.  build.sh imposes the
6731.105Swiz               restriction that the argument to the -M option must not begin
6741.126Swiz               with a "$" (dollar sign) character; otherwise it would be too
6751.104Swiz               difficult to determine whether the value is an absolute or a
6761.104Swiz               relative path.  If the directory does not already exist,
6771.105Swiz               build.sh will create it.
6781.18Slukem
6791.109Sapb     -m mach   Set the value of MACHINE to mach, unless the mach argument is
6801.109Sapb               an alias that refers to a MACHINE/MACHINE_ARCH pair, in which
6811.109Sapb               case both MACHINE and MACHINE_ARCH are set from the alias.
6821.109Sapb               Such aliases are interpreted entirely by build.sh; they are not
6831.109Sapb               used by any other part of the build system.  The MACHINE_ARCH
6841.109Sapb               setting implied by mach will override any value of MACHINE_ARCH
6851.109Sapb               in the process environment, but will not override a value set
6861.109Sapb               by the -a option.  All cross builds require -m, but if unset on
6871.109Sapb               a NetBSD host, the host's value of MACHINE will be detected and
6881.109Sapb               used automatically.
6891.109Sapb
6901.109Sapb               See the list-arch operation for a way to get a list of valid
6911.109Sapb               MACHINE and MACHINE_ARCH settings.
6921.1Stv
6931.105Swiz     -N noiselevel
6941.126Swiz               Set the "noisyness" level of the build, by setting MAKEVERBOSE
6951.126Swiz               to noiselevel.
6961.45Slukem
6971.105Swiz     -n        Show the commands that would be executed by build.sh, but do
6981.126Swiz               not make any changes.  This is similar in concept to "make -n".
6991.1Stv
7001.105Swiz     -O obj    Create an appropriate transform macro for MAKEOBJDIR that will
7011.105Swiz               place the built object files under obj.  Unsets
7021.105Swiz               MAKEOBJDIRPREFIX.
7031.88Sapb
7041.126Swiz               For instance, a setting of "-O /usr/obj" will place build-time
7051.126Swiz               files under /usr/obj/bin, /usr/obj/lib, /usr/obj/usr.bin, and
7061.126Swiz               so forth.
7071.88Sapb
7081.88Sapb               If a relative path is specified, it will be converted to an
7091.105Swiz               absolute path before being used.  build.sh imposes the
7101.105Swiz               restriction that the argument to the -O option must not contain
7111.126Swiz               a "$" (dollar sign) character.  If the directory does not
7121.105Swiz               already exist, build.sh will create it.
7131.73Sapb
7141.105Swiz               In normal use, exactly one of the -M or -O options should be
7151.105Swiz               specified.  If neither -M nor -O is specified, then a default
7161.78Sapb               object directory will be chosen according to rules in
7171.78Sapb               <bsd.obj.mk>.  Relying on this default is not recommended
7181.78Sapb               because it is determined by complex rules that are influenced
7191.78Sapb               by the values of several variables and by the location of the
7201.78Sapb               source directory.
7211.18Slukem
7221.151Slukem               Note: Placing the obj directory location outside of the default
7231.151Slukem               source tree hierarchy makes it easier to manually clear out old
7241.151Slukem               files in the event the "make cleandir" operation is unable to
7251.151Slukem               do so.  (See CAVEATS below.)
7261.151Slukem
7271.151Slukem               Note: The use of one of -M or -O is the only means of building
7281.151Slukem               multiple machine architecture userlands from the same source
7291.151Slukem               tree without cleaning between builds (in which case, one would
7301.151Slukem               specify distinct obj locations for each).
7311.109Sapb
7321.126Swiz     -o        Set the value of MKOBJDIRS to "no".  Otherwise, it will be
7331.126Swiz               automatically set to "yes".  This default is opposite to the
7341.105Swiz               behaviour when not using build.sh.
7351.1Stv
7361.148Slukem     -P        Set the value of MKREPRO and MKREPRO_TIMESTAMP to the latest
7371.148Slukem               source CVS timestamp for reproducible builds.
7381.148Slukem
7391.105Swiz     -R rel    Set the value of RELEASEDIR to rel.  If a relative path is
7401.59Sapb               specified, it will be converted to an absolute path before
7411.59Sapb               being used.
7421.18Slukem
7431.105Swiz     -r        Remove the contents of DESTDIR and TOOLDIR before building
7441.1Stv               (provides a clean starting point).  This will skip deleting
7451.105Swiz               DESTDIR if building on a native system to the root directory.
7461.1Stv
7471.105Swiz     -S seed   Change the value of BUILDSEED to seed.  This should rarely be
7481.72Sperry               necessary.
7491.72Sperry
7501.105Swiz     -T tools  Set the value of TOOLDIR to tools.  If a relative path is
7511.104Swiz               specified, it will be converted to an absolute path before
7521.126Swiz               being used.  If set, the bootstrap "make" will only be rebuilt
7531.126Swiz               if the source files for make(1) have changed.
7541.104Swiz
7551.105Swiz     -U        Set MKUNPRIVED=yes.
7561.104Swiz
7571.105Swiz     -u        Set MKUPDATE=yes.
7581.104Swiz
7591.105Swiz     -V var=[value]
7601.105Swiz               Set the environment variable var to an optional value.  This is
7611.159Slukem               propagated to the nbmake-MACHINE wrapper script.
7621.105Swiz
7631.105Swiz     -w wrapper
7641.159Slukem               Create the nbmake-MACHINE wrapper script (see below) in a
7651.159Slukem               custom location, specified by wrapper.  This allows, for
7661.159Slukem               instance, to place the wrapper script in PATH automatically.
7671.151Slukem
7681.151Slukem               Note: wrapper is the full name of the file, not just a
7691.151Slukem               directory name.  If a relative path is specified, it will be
7701.151Slukem               converted to an absolute path before being used.
7711.1Stv
7721.105Swiz     -X x11src
7731.105Swiz               Set the value of X11SRCDIR to x11src.  If a relative path is
7741.59Sapb               specified, it will be converted to an absolute path before
7751.59Sapb               being used.
7761.49Slukem
7771.105Swiz     -x        Set MKX11=yes.
7781.49Slukem
7791.105Swiz     -Z var    Unset ("zap") the environment variable var.  This is propagated
7801.159Slukem               to the nbmake-MACHINE wrapper script.
7811.40Slukem
7821.148Slukem     -?        Show a help message, and exit.
7831.148Slukem
7841.105Swiz   The "nbmake-MACHINE" wrapper script
7851.159Slukem     If using the build.sh script to build NetBSD, a nbmake-MACHINE wrapper
7861.159Slukem     script will be created in TOOLDIR/bin upon the first build to assist in
7871.159Slukem     building subtrees on a cross-compile host.
7881.159Slukem
7891.159Slukem     The nbmake-MACHINE wrapper script can be invoked in lieu of make(1), and
7901.159Slukem     will instead call the up-to-date version of "nbmake" installed into
7911.159Slukem     TOOLDIR/bin with several key variables pre-set, including MACHINE,
7921.159Slukem     MACHINE_ARCH, and TOOLDIR.  nbmake-MACHINE will also set variables
7931.159Slukem     specified with -V, and unset variables specified with -Z.  Note that by
7941.159Slukem     default these variables will not override mk.conf(5); see "make"
7951.159Slukem     variables for more details.
7961.40Slukem
7971.159Slukem     This wrapper script can be symlinked into a directory listed in PATH, or
7981.159Slukem     called with an absolute path.
7991.1Stv
8001.105SwizEXAMPLES
8011.151Slukem     1.   % ./build.sh [OPTIONS] tools kernel=GENERIC
8021.28Slukem
8031.28Slukem          Build a new toolchain, and use the new toolchain to configure and
8041.28Slukem          build a new GENERIC kernel.
8051.28Slukem
8061.151Slukem     2.   % ./build.sh [OPTIONS] -U distribution
8071.28Slukem
8081.105Swiz          Using unprivileged mode, build a complete distribution to a DESTDIR
8091.148Slukem          directory that build.sh selects (and will show).
8101.28Slukem
8111.151Slukem     3.   # ./build.sh [OPTIONS] -U install=/
8121.28Slukem
8131.105Swiz          As root, install to / the distribution that was built by example 2.
8141.105Swiz          Even though this is run as root, -U is required so that the
8151.105Swiz          permissions stored in DESTDIR/METALOG are correctly applied to the
8161.105Swiz          files as they're copied to /.
8171.28Slukem
8181.151Slukem     4.   % ./build.sh [OPTIONS] -U -u release
8191.28Slukem
8201.105Swiz          Using unprivileged mode, build a complete release to DESTDIR and
8211.148Slukem          RELEASEDIR directories that build.sh selects (and will show).
8221.126Swiz          MKUPDATE=yes (-u) is set to prevent the "make cleandir", so that if
8231.126Swiz          this is run after example 2, it doesn't need to redo that portion of
8241.126Swiz          the release build.
8251.16Slukem
8261.105SwizSEE ALSO
8271.155Slukem     config(1), ctags(1), ex(1), gzip(1), lint(1), make(1), mandoc(1),
8281.155Slukem     mkisofs(1), nroff(1), vi(1), mk.conf(5), atf(7), hier(7), mdoc(7),
8291.155Slukem     release(7), etcupdate(8), installboot(8), mount(8), postinstall(8),
8301.151Slukem     sysinst(8), pkgsrc/sysutils/cdrtools
8311.1Stv
8321.155Slukem     Note: The NetBSD manual pages are also available at
8331.155Slukem     https://man.netbsd.org
8341.155Slukem
8351.105SwizHISTORY
8361.105Swiz     The build.sh based build scheme was introduced for NetBSD 1.6 as
8371.105Swiz     USE_NEW_TOOLCHAIN, and re-worked to TOOLCHAIN_MISSING after that.
8381.1Stv
8391.109SapbCAVEATS
8401.109Sapb     After significant updates to third-party components in the source tree,
8411.126Swiz     the "make cleandir" operation may be insufficient to clean out old files
8421.126Swiz     in object directories.  Instead, one may have to manually remove the
8431.126Swiz     files.  Consult the UPDATING file for notices concerning this.
8441.109Sapb
8451.159SlukemNetBSD                           July 21, 2023                          NetBSD
846