BUILDING revision 1.157 1 1.104 wiz BUILDING(8) System Manager's Manual BUILDING(8)
2 1.1 tv
3 1.105 wiz NAME
4 1.150 lukem BUILDING - Procedure for building NetBSD from source code
5 1.1 tv
6 1.105 wiz REQUIREMENTS
7 1.1 tv NetBSD is designed to be buildable on most POSIX-compliant host systems.
8 1.105 wiz The basic build procedure is the same whether compiling natively (on the
9 1.105 wiz same NetBSD architecture) or cross compiling (on another architecture or
10 1.1 tv OS).
11 1.1 tv
12 1.157 lukem This source tree contains the build.sh shell script which supports both
13 1.157 lukem native and cross builds of NetBSD.
14 1.157 lukem
15 1.126 wiz This source tree contains a special subtree, "tools", which uses the host
16 1.126 wiz system to create a build toolchain for the target architecture. The host
17 1.126 wiz system must have at least C and C++ compilers in order to create the
18 1.157 lukem toolchain (make(1) is not required); all other tools (including make(1)
19 1.157 lukem as nbmake) are created as part of the NetBSD build process. (See the
20 1.157 lukem Environment variables section below if you need to override or manually
21 1.157 lukem select your compilers.)
22 1.1 tv
23 1.155 lukem Note: Within this document, cross-references to manual pages are to the
24 1.155 lukem NetBSD manual pages, not the host system manual pages. The mdoc(7)
25 1.155 lukem source to the NetBSD manual pages can be found within the source tree,
26 1.155 lukem and these and can be formatted with mandoc(1) or nroff(1) if those are
27 1.155 lukem available on the host system. The NetBSD manual pages are also available
28 1.155 lukem at https://man.netbsd.org
29 1.155 lukem
30 1.105 wiz FILES
31 1.105 wiz Source tree layout
32 1.151 lukem BUILDING This document (in plaintext). Generated from
33 1.151 lukem doc/BUILDING.mdoc.
34 1.48 jmc
35 1.105 wiz Makefile The main Makefile for NetBSD; should only be run for
36 1.44 lukem native builds with an appropriately up-to-date version of
37 1.142 andvar NetBSD make(1). Intended for expert use with knowledge of
38 1.129 sevan its shortcomings, it has been superseded by the build.sh
39 1.129 sevan shell script as the recommended means for building NetBSD.
40 1.1 tv
41 1.105 wiz UPDATING Special notes for updating from an earlier revision of
42 1.1 tv NetBSD. It is important to read this file before every
43 1.1 tv build of an updated source tree.
44 1.1 tv
45 1.105 wiz build.sh Bourne-compatible shell script used for building the host
46 1.1 tv build tools and the NetBSD system from scratch. Can be
47 1.1 tv used for both native and cross builds, and should be used
48 1.129 sevan instead of make(1) as it performs additional checks to
49 1.129 sevan prevent common issues going undetected, such as building
50 1.129 sevan with an outdated version of make(1).
51 1.1 tv
52 1.105 wiz crypto/dist/, dist/, gnu/dist/
53 1.104 wiz Sources imported verbatim from third parties, without
54 1.104 wiz mangling the existing build structure. Other source trees
55 1.126 wiz in bin through usr.sbin use the NetBSD make(1) "reachover"
56 1.126 wiz Makefile semantics when building these programs for a
57 1.126 wiz native host.
58 1.104 wiz
59 1.151 lukem distrib/, etc/
60 1.151 lukem Sources for items used when making a full release
61 1.151 lukem snapshot, such as files installed in DESTDIR/etc on the
62 1.151 lukem destination system, boot media, and release notes.
63 1.151 lukem
64 1.151 lukem doc/BUILDING.mdoc
65 1.155 lukem The source to this document, in mdoc(7) format. Used to
66 1.155 lukem generate BUILDING.
67 1.151 lukem
68 1.106 wiz external, sys/external
69 1.106 wiz Sources and build infrastructure for components imported
70 1.106 wiz (mostly) unchanged from upstream maintainers, sorted by
71 1.106 wiz applicable license. This is (slowly) replacing the
72 1.106 wiz crypto/dist, dist, and gnu/dist directories.
73 1.106 wiz
74 1.151 lukem external/mit/xorg/
75 1.151 lukem "Reachover" build structure for modular Xorg; the source
76 1.151 lukem is in X11SRCDIR.
77 1.1 tv
78 1.157 lukem mk.conf Optional source tree specific mk.conf(5), used (if
79 1.157 lukem present) instead of /etc/mk.conf unless MAKECONF is
80 1.157 lukem defined.
81 1.157 lukem
82 1.157 lukem Note: Not part of the NetBSD source repository.
83 1.157 lukem
84 1.151 lukem regress/, tests/
85 1.70 dholland Regression test harness. Can be cross-compiled, but only
86 1.105 wiz run natively. tests/ uses the atf(7) test framework;
87 1.105 wiz regress/ contains older tests that have not yet been
88 1.70 dholland migrated to atf(7).
89 1.1 tv
90 1.105 wiz sys/ NetBSD kernel sources.
91 1.1 tv
92 1.126 wiz tools/ "Reachover" build structure for the host build tools.
93 1.104 wiz This has a special method of determining out-of-date
94 1.104 wiz status.
95 1.1 tv
96 1.151 lukem tools/compat/README
97 1.151 lukem Special notes for cross-hosting a NetBSD build on non-
98 1.151 lukem NetBSD platforms.
99 1.151 lukem
100 1.151 lukem Other directories including bin/ ... usr.sbin/
101 1.1 tv Sources to the NetBSD userland (non-kernel) programs. If
102 1.1 tv any of these directories are missing, they will be skipped
103 1.1 tv during the build.
104 1.1 tv
105 1.105 wiz Build tree layout
106 1.156 lukem The NetBSD build tree is described in hier(7) (whose mdoc(7) source is in
107 1.155 lukem share/man/man7/hier.7), and the release layout is described in release(7)
108 1.156 lukem (whose mdoc(7) source is in share/man/man7/release.7).
109 1.1 tv
110 1.105 wiz CONFIGURATION
111 1.105 wiz Environment variables
112 1.8 lukem Several environment variables control the behaviour of NetBSD builds.
113 1.8 lukem
114 1.152 lukem HOST_CC Path name to C compiler used to create the toolchain.
115 1.58 apb
116 1.155 lukem Default: "cc".
117 1.155 lukem
118 1.152 lukem HOST_CFLAGS Flags passed to the host C compiler.
119 1.53 reed
120 1.155 lukem Default: "-O".
121 1.155 lukem
122 1.155 lukem HOST_CPPFLAGS Flags passed to the host C/C++ pre-processor.
123 1.155 lukem
124 1.155 lukem Default: Unset.
125 1.155 lukem
126 1.152 lukem HOST_CXX Path name to C++ compiler used to create the toolchain.
127 1.137 uwe
128 1.155 lukem Default: Unset, but defaults to "c++" where required.
129 1.155 lukem
130 1.152 lukem HOST_CXXFLAGS Flags passed to the host C++ compiler.
131 1.53 reed
132 1.155 lukem Default: Unset.
133 1.155 lukem
134 1.152 lukem HOST_SH Path name to a shell available on the host system and
135 1.152 lukem suitable for use during the build. The NetBSD build
136 1.152 lukem system requires a modern Bourne-like shell with POSIX-
137 1.152 lukem compliant features, and also requires support for the
138 1.152 lukem "local" keyword to declare local variables in shell
139 1.152 lukem functions (which is a widely-implemented but non-
140 1.152 lukem standardised feature).
141 1.152 lukem
142 1.152 lukem Depending on the host system, a suitable shell may be
143 1.152 lukem /bin/sh, /usr/xpg4/bin/sh, /bin/ksh (provided it is a
144 1.152 lukem variant of ksh that supports the "local" keyword, such as
145 1.152 lukem ksh88, but not ksh93), or /usr/local/bin/bash.
146 1.152 lukem
147 1.152 lukem Most parts of the build require HOST_SH to be an absolute
148 1.152 lukem path; however, build.sh allows it to be a simple command
149 1.152 lukem name, which will be converted to an absolute path by
150 1.152 lukem searching the PATH.
151 1.137 uwe
152 1.155 lukem Default: "sh".
153 1.155 lukem
154 1.147 lukem INSTALLBOOT_UBOOT_PATHS
155 1.152 lukem A colon-separated list of search paths used by
156 1.152 lukem installboot(8) to find U-Boot packages.
157 1.147 lukem
158 1.155 lukem Default: Unset.
159 1.155 lukem
160 1.152 lukem MACHINE Machine type, e.g., "macppc".
161 1.8 lukem
162 1.155 lukem Default: Unset.
163 1.155 lukem
164 1.152 lukem MACHINE_ARCH Machine architecture, e.g., "powerpc".
165 1.8 lukem
166 1.155 lukem Default: Unset.
167 1.155 lukem
168 1.152 lukem MAKE Path name to invoke make(1) as.
169 1.8 lukem
170 1.155 lukem Default: "make".
171 1.155 lukem
172 1.152 lukem MAKECONF The name of the make(1) configuration file. See "make"
173 1.152 lukem variables and mk.conf(5).
174 1.151 lukem
175 1.152 lukem Note: Only settable in the process environment.
176 1.151 lukem
177 1.157 lukem Default: "/etc/mk.conf", although build.sh will set the
178 1.157 lukem default to the full path to mk.conf if the latter is
179 1.157 lukem present in the same directory as build.sh.
180 1.151 lukem
181 1.152 lukem MAKEFLAGS Flags to invoke make(1) with.
182 1.151 lukem
183 1.152 lukem Note: build.sh ignores the value of MAKEFLAGS passed in
184 1.152 lukem the environment, but allows MAKEFLAGS to be set via the
185 1.152 lukem -V option.
186 1.151 lukem
187 1.155 lukem Default: "-X" on systems with a small ARG_MAX (Cygwin,
188 1.155 lukem Darwin, FreeBSD); otherwise unset.
189 1.155 lukem
190 1.152 lukem MAKEOBJDIR Directory to use as the .OBJDIR for the current
191 1.152 lukem directory. The value is subjected to variable expansion
192 1.152 lukem by make(1). Typical usage is to set this variable to a
193 1.152 lukem value involving the use of `${.CURDIR:S...}' or
194 1.152 lukem `${.CURDIR:C...}', to derive the value of .OBJDIR from
195 1.152 lukem the value of .CURDIR. Used only if MAKEOBJDIRPREFIX is
196 1.152 lukem not defined.
197 1.151 lukem
198 1.152 lukem Note: MAKEOBJDIR can be provided only in the environment
199 1.152 lukem or via the -O flag of build.sh; it cannot usefully be set
200 1.152 lukem inside a Makefile, including in mk.conf(5) or MAKECONF.
201 1.151 lukem
202 1.155 lukem Default: Unset.
203 1.155 lukem
204 1.151 lukem MAKEOBJDIRPREFIX
205 1.152 lukem Top level directory of the object directory tree. The
206 1.152 lukem value is subjected to variable expansion by make(1).
207 1.152 lukem build.sh will create the ${MAKEOBJDIRPREFIX} directory if
208 1.152 lukem necessary, but if make(1) is used without build.sh, then
209 1.152 lukem rules in <bsd.obj.mk> will abort the build if the
210 1.152 lukem ${MAKEOBJDIRPREFIX} directory does not exist. If the
211 1.152 lukem value is defined and valid, then
212 1.152 lukem ${MAKEOBJDIRPREFIX}/${.CURDIR} is used as the .OBJDIR for
213 1.152 lukem the current directory. The current directory may be read
214 1.152 lukem only.
215 1.152 lukem
216 1.152 lukem Note: MAKEOBJDIRPREFIX can be provided only in the
217 1.152 lukem environment or via the -M flag of build.sh; it cannot
218 1.152 lukem usefully be set inside a Makefile, including in
219 1.152 lukem mk.conf(5) or MAKECONF.
220 1.152 lukem
221 1.155 lukem Default: Unset.
222 1.155 lukem
223 1.152 lukem TMPDIR Top-level directory to store temporary directories used
224 1.152 lukem by build.sh before paths to other directories such as
225 1.152 lukem .OBJDIR can be determined.
226 1.149 lukem
227 1.152 lukem Note: Must support execution of binaries. I.e., without
228 1.152 lukem mount(8)'s -o noexec option.
229 1.149 lukem
230 1.152 lukem Default: "/tmp".
231 1.149 lukem
232 1.105 wiz "make" variables
233 1.156 lukem Variables that control the behavior of NetBSD builds are documented in
234 1.156 lukem mk.conf(5) (whose mdoc(7) source is in share/man/man5/mk.conf.5).
235 1.45 lukem
236 1.155 lukem Unless otherwise specified, these variables may be set in either the
237 1.155 lukem process environment or the make(1) configuration file mk.conf(5)
238 1.155 lukem specified by MAKECONF.
239 1.49 lukem
240 1.105 wiz BUILDING
241 1.105 wiz "make" command line options
242 1.52 jmc This is not a summary of all the options available to make(1); only the
243 1.52 jmc options used most frequently with NetBSD builds are listed here.
244 1.1 tv
245 1.105 wiz -j njob Run up to njob make(1) subjobs in parallel. Makefiles should
246 1.52 jmc use .WAIT or have explicit dependencies as necessary to
247 1.74 apb enforce build ordering.
248 1.10 sommerfe
249 1.105 wiz -m dir Specify the default directory for searching for system
250 1.104 wiz Makefile segments, mainly the <bsd.*.mk> files. When building
251 1.104 wiz any full NetBSD source tree, this should be set to the
252 1.126 wiz "share/mk" directory in the source tree. This is set
253 1.104 wiz automatically when building from the top level, or when using
254 1.105 wiz build.sh.
255 1.1 tv
256 1.148 lukem -n Show the commands that would have been executed, but do not
257 1.1 tv actually execute them. This will still cause recursion to
258 1.1 tv take place.
259 1.1 tv
260 1.148 lukem -V var Show make(1)'s idea of the value of var. Does not build any
261 1.1 tv targets.
262 1.1 tv
263 1.105 wiz var=value Set the variable var to value, overriding any setting
264 1.105 wiz specified by the process environment, the MAKECONF
265 1.104 wiz configuration file, or the system Makefile segments.
266 1.1 tv
267 1.105 wiz "make" targets
268 1.1 tv These default targets may be built by running make(1) in any subtree of
269 1.1 tv the NetBSD source code. It is recommended that none of these be used
270 1.126 wiz from the top level Makefile; as a specific exception, "make obj" and
271 1.126 wiz "make cleandir" are useful in that context.
272 1.1 tv
273 1.105 wiz all Build programs, libraries, and preformatted documentation.
274 1.1 tv
275 1.105 wiz clean Remove program and library object code files.
276 1.1 tv
277 1.105 wiz cleandir Same as clean, but also remove preformatted documentation,
278 1.126 wiz dependency files generated by "make depend", and any other
279 1.49 lukem files known to be created at build time.
280 1.1 tv
281 1.105 wiz depend Create dependency files (.depend) containing more detailed
282 1.44 lukem information about the dependencies of source code on header
283 1.1 tv files. Allows programs to be recompiled automatically when a
284 1.1 tv dependency changes.
285 1.1 tv
286 1.126 wiz dependall Does a "make depend" immediately followed by a "make all".
287 1.10 sommerfe This improves cache locality of the build since both passes
288 1.10 sommerfe read the source files in their entirety.
289 1.1 tv
290 1.105 wiz distclean Synonym for cleandir.
291 1.49 lukem
292 1.105 wiz includes Build and install system header files. Typically needed
293 1.44 lukem before any system libraries or programs can be built.
294 1.1 tv
295 1.105 wiz install Install programs, libraries, and documentation into DESTDIR.
296 1.105 wiz Few files will be installed to DESTDIR/dev, DESTDIR/etc,
297 1.105 wiz DESTDIR/root or DESTDIR/var in order to prevent user supplied
298 1.27 lukem configuration data from being overwritten.
299 1.1 tv
300 1.105 wiz lint Run lint(1) against the C source code, where appropriate, and
301 1.1 tv generate system-installed lint libraries.
302 1.1 tv
303 1.105 wiz obj Create object directories to be used for built files, instead
304 1.1 tv of building directly in the source tree.
305 1.1 tv
306 1.105 wiz tags Create ctags(1) searchable function lists usable by the ex(1)
307 1.1 tv and vi(1) text editors.
308 1.1 tv
309 1.105 wiz "make" targets for the top level
310 1.1 tv Additional make(1) targets are usable specifically from the top source
311 1.1 tv level to facilitate building the entire NetBSD source tree.
312 1.1 tv
313 1.105 wiz build Build the entire NetBSD system (except the kernel). This
314 1.68 reed orders portions of the source tree such that prerequisites
315 1.68 reed will be built in the proper order.
316 1.20 lukem
317 1.126 wiz distribution Do a "make build", and then install a full distribution
318 1.105 wiz (which does not include a kernel) into DESTDIR, including
319 1.105 wiz files in DESTDIR/dev, DESTDIR/etc, DESTDIR/root and
320 1.105 wiz DESTDIR/var.
321 1.20 lukem
322 1.126 wiz buildworld As per "make distribution", except that it ensures that
323 1.105 wiz DESTDIR is not the root directory.
324 1.20 lukem
325 1.105 wiz installworld Install the distribution from DESTDIR to INSTALLWORLDDIR,
326 1.70 dholland which defaults to the root directory. Ensures that
327 1.105 wiz INSTALLWORLDDIR is not the root directory if cross
328 1.104 wiz compiling.
329 1.20 lukem
330 1.105 wiz The INSTALLSETS environment variable may be set to a space-
331 1.95 apb separated list of distribution sets to be installed. By
332 1.126 wiz default, all sets except "etc" and "xetc" are installed, so
333 1.126 wiz most files in INSTALLWORLDDIR/etc will not be installed or
334 1.126 wiz modified.
335 1.61 apb
336 1.105 wiz Note: Before performing this operation with
337 1.105 wiz INSTALLWORLDDIR=/, it is highly recommended that you
338 1.61 apb upgrade your kernel and reboot. After performing this
339 1.61 apb operation, it is recommended that you use etcupdate(8) to
340 1.105 wiz update files in INSTALLWORLDDIR/etc, and postinstall(8) to
341 1.102 abs check for or fix inconsistencies.
342 1.20 lukem
343 1.105 wiz sets Create distribution sets from DESTDIR into
344 1.105 wiz RELEASEDIR/RELEASEMACHINEDIR/binary/sets. Should be run
345 1.126 wiz after "make distribution", as "make build" alone does not
346 1.126 wiz install all of the required files.
347 1.25 lukem
348 1.105 wiz sourcesets Create source sets of the source tree into
349 1.105 wiz RELEASEDIR/source/sets.
350 1.31 lukem
351 1.105 wiz syspkgs Create syspkgs from DESTDIR into
352 1.105 wiz RELEASEDIR/RELEASEMACHINEDIR/binary/syspkgs. Should be run
353 1.126 wiz after "make distribution", as "make build" alone does not
354 1.126 wiz install all of the required files.
355 1.54 apb
356 1.126 wiz release Do a "make distribution", build kernels, distribution
357 1.126 wiz media, and install sets (this as per "make sets"), and then
358 1.126 wiz package the system into a standard release layout as
359 1.105 wiz described by release(7). This requires that RELEASEDIR be
360 1.25 lukem set (see above).
361 1.1 tv
362 1.105 wiz iso-image Create a NetBSD installation CD-ROM image in the
363 1.114 snj RELEASEDIR/images directory. The CD-ROM file system will
364 1.114 snj have a layout as described in release(7).
365 1.62 apb
366 1.62 apb For most machine types, the CD-ROM will be bootable, and
367 1.104 wiz will automatically run the sysinst(8) menu-based
368 1.104 wiz installation program, which can be used to install or
369 1.104 wiz upgrade a NetBSD system. Bootable CD-ROMs also contain
370 1.104 wiz tools that may be useful in repairing a damaged NetBSD
371 1.104 wiz installation.
372 1.62 apb
373 1.126 wiz Before "make iso-image" is attempted, RELEASEDIR must be
374 1.126 wiz populated by "make release" or equivalent.
375 1.62 apb
376 1.151 lukem Note: Other, smaller, CD-ROM images may be created in the
377 1.151 lukem RELEASEDIR/RELEASEMACHINEDIR/installation/cdrom directory
378 1.151 lukem by "make release". These smaller images usually contain
379 1.151 lukem the same tools as the larger images in RELEASEDIR/images,
380 1.151 lukem but do not contain additional content such as the
381 1.151 lukem distribution sets.
382 1.62 apb
383 1.151 lukem Note: The mac68k port still uses an older method of
384 1.104 wiz creating CD-ROM images. This requires the mkisofs(1)
385 1.104 wiz utility, which is not part of NetBSD, but which can be
386 1.105 wiz installed from pkgsrc/sysutils/cdrtools.
387 1.56 apb
388 1.105 wiz iso-image-source
389 1.64 jnemeth Create a NetBSD installation CD-ROM image in the
390 1.114 snj RELEASEDIR/images directory. The CD-ROM file system will
391 1.114 snj have a layout as described in release(7). It will have top
392 1.64 jnemeth level directories for the machine type and source.
393 1.64 jnemeth
394 1.64 jnemeth For most machine types, the CD-ROM will be bootable, and
395 1.104 wiz will automatically run the sysinst(8) menu-based
396 1.104 wiz installation program, which can be used to install or
397 1.104 wiz upgrade a NetBSD system. Bootable CD-ROMs also contain
398 1.104 wiz tools that may be useful in repairing a damaged NetBSD
399 1.104 wiz installation.
400 1.64 jnemeth
401 1.126 wiz Before "make iso-image-source" is attempted, RELEASEDIR
402 1.126 wiz must be populated by "make sourcesets release" or
403 1.104 wiz equivalent.
404 1.64 jnemeth
405 1.151 lukem Note: Other, smaller, CD-ROM images may be created in the
406 1.151 lukem RELEASEDIR/RELEASEMACHINEDIR/installation/cdrom directory
407 1.151 lukem by "make release". These smaller images usually contain
408 1.151 lukem the same tools as the larger images in RELEASEDIR/images,
409 1.151 lukem but do not contain additional content such as the
410 1.151 lukem distribution sets.
411 1.64 jnemeth
412 1.151 lukem Note: The mac68k port still uses an older method of
413 1.104 wiz creating CD-ROM images. This requires the mkisofs(1)
414 1.104 wiz utility, which is not part of NetBSD, but which can be
415 1.105 wiz installed from pkgsrc/sysutils/cdrtools.
416 1.64 jnemeth
417 1.105 wiz install-image
418 1.98 apb Create a bootable NetBSD installation disk image in the
419 1.127 plunky RELEASEDIR/images directory. The installation disk image
420 1.127 plunky is suitable for copying to bootable USB flash memory
421 1.127 plunky sticks, etc., for machines which are able to boot from such
422 1.127 plunky devices. The file system in the bootable disk image will
423 1.127 plunky have a layout as described in release(7).
424 1.97 tsutsui
425 1.97 tsutsui The installation image is bootable, and will automatically
426 1.97 tsutsui run the sysinst(8) menu-based installation program, which
427 1.98 apb can be used to install or upgrade a NetBSD system. The
428 1.98 apb image also contains tools that may be useful in repairing a
429 1.98 apb damaged NetBSD installation.
430 1.97 tsutsui
431 1.126 wiz Before "make install-image" is attempted, RELEASEDIR must
432 1.126 wiz be populated by "make release" or equivalent. The build
433 1.126 wiz must have been performed with MKUNPRIVED=yes because "make
434 1.126 wiz install-image" relies on information in DESTDIR/METALOG.
435 1.97 tsutsui
436 1.120 pgoyette live-image Create NetBSD live images in the RELEASEDIR/images
437 1.104 wiz directory. The live image contains all necessary files to
438 1.104 wiz boot NetBSD up to multi-user mode, including all files
439 1.104 wiz which should be extracted during installation, NetBSD
440 1.104 wiz disklabel, bootloaders, etc.
441 1.104 wiz
442 1.104 wiz The live image is suitable for use as a disk image in
443 1.104 wiz virtual machine environments such as QEMU, and also useful
444 1.104 wiz to boot NetBSD from a USB flash memory stick on a real
445 1.98 apb machine, without the need for installation.
446 1.97 tsutsui
447 1.126 wiz Before "make live-image" is attempted, RELEASEDIR must be
448 1.126 wiz populated by "make release" or equivalent. The build must
449 1.126 wiz have been performed with MKUNPRIVED=yes because "make
450 1.126 wiz install-image" relies on information in DESTDIR/METALOG.
451 1.97 tsutsui
452 1.105 wiz regression-tests
453 1.20 lukem Can only be run after building the regression tests in the
454 1.126 wiz directory "regress". Runs those compiled regression tests
455 1.151 lukem on the local host.
456 1.151 lukem
457 1.151 lukem Note: Most tests are now managed instead using atf(7); this
458 1.151 lukem target should probably run those as well but currently does
459 1.151 lukem not.
460 1.1 tv
461 1.105 wiz The "build.sh" script
462 1.96 apb This script file is a shell script designed to build the entire NetBSD
463 1.104 wiz system on any host with a suitable modern shell and some common
464 1.105 wiz utilities. The required shell features are described under the HOST_SH
465 1.104 wiz variable.
466 1.96 apb
467 1.96 apb If a host system's default shell does support the required features, then
468 1.96 apb we suggest that you explicitly specify a suitable shell using a command
469 1.96 apb like
470 1.96 apb
471 1.105 wiz /path/to/suitable/shell build.sh [options]
472 1.96 apb
473 1.105 wiz The above command will usually enable build.sh to automatically set
474 1.105 wiz HOST_SH=/path/to/suitable/shell, but if that fails, then the following
475 1.96 apb set of commands may be used instead:
476 1.96 apb
477 1.105 wiz HOST_SH=/path/to/suitable/shell
478 1.96 apb export HOST_SH
479 1.105 wiz ${HOST_SH} build.sh [options]
480 1.96 apb
481 1.105 wiz If build.sh detects that it is being executed under an unsuitable shell,
482 1.148 lukem it attempts to exec a suitable shell instead, or shows an error message.
483 1.105 wiz If HOST_SH is not set explicitly, then build.sh sets a default using
484 1.96 apb heuristics dependent on the host platform, or from the shell under which
485 1.105 wiz build.sh is executed (if that can be determined), or using the first copy
486 1.105 wiz of sh found in PATH.
487 1.1 tv
488 1.1 tv All cross-compile builds, and most native builds, of the entire system
489 1.126 wiz should make use of build.sh rather than just running "make". This way,
490 1.104 wiz the make(1) program will be bootstrapped properly, in case the host
491 1.126 wiz system has an older or incompatible "make" program.
492 1.1 tv
493 1.105 wiz When compiling the entire system via build.sh, many make(1) variables are
494 1.1 tv set for you in order to help encapsulate the build process. In the list
495 1.105 wiz of options below, variables that are automatically set by build.sh are
496 1.1 tv noted where applicable.
497 1.1 tv
498 1.105 wiz The following operations are supported by build.sh:
499 1.22 lukem
500 1.126 wiz build Build the system as per "make build". Before the main part
501 1.126 wiz of the build commences, this command runs the obj operation
502 1.126 wiz (unless the -o option is given), "make cleandir" (unless
503 1.126 wiz the -u option is given), and the tools operation.
504 1.22 lukem
505 1.126 wiz distribution Build a full distribution as per "make distribution". This
506 1.126 wiz command first runs the build operation.
507 1.22 lukem
508 1.126 wiz release Build a full release as per "make release". This command
509 1.105 wiz first runs the distribution operation.
510 1.22 lukem
511 1.148 lukem help Show a help message, and exit.
512 1.148 lukem
513 1.105 wiz makewrapper Create the nbmake-MACHINE wrapper. This operation is
514 1.104 wiz automatically performed for any of the other operations.
515 1.23 lukem
516 1.126 wiz cleandir Perform "make cleandir".
517 1.75 lukem
518 1.126 wiz obj Perform "make obj".
519 1.22 lukem
520 1.105 wiz tools Build and install the host tools from src/tools. This
521 1.126 wiz command will first run "make obj" and "make cleandir" in
522 1.126 wiz the tools subdirectory unless the -o or -u options
523 1.104 wiz (respectively) are given.
524 1.22 lukem
525 1.126 wiz install=idir Install the contents of DESTDIR to idir, using "make
526 1.151 lukem installworld".
527 1.151 lukem
528 1.151 lukem Note: Files that are part of the "etc" or "xetc" sets will
529 1.151 lukem not be installed, unless overridden by the INSTALLSETS
530 1.151 lukem environment variable.
531 1.35 lukem
532 1.105 wiz kernel=kconf Build a new kernel. The kconf argument is the name of a
533 1.105 wiz configuration file suitable for use by config(1). If kconf
534 1.22 lukem does not contain any `/' characters, the configuration file
535 1.105 wiz is expected to be found in the KERNCONFDIR directory, which
536 1.105 wiz is typically sys/arch/MACHINE/conf. The new kernel will be
537 1.105 wiz built in a subdirectory of KERNOBJDIR, which is typically
538 1.105 wiz sys/arch/MACHINE/compile or an associated object directory.
539 1.70 dholland
540 1.105 wiz This command does not imply the tools command; run the
541 1.105 wiz tools command first unless it is certain that the tools
542 1.70 dholland already exist and are up to date.
543 1.70 dholland
544 1.126 wiz This command will run "make cleandir" on the kernel in
545 1.105 wiz question first unless the -u option is given.
546 1.22 lukem
547 1.106 wiz kernel.gdb=kconf
548 1.106 wiz Build a new kernel with debug information. Similar to the
549 1.106 wiz above kernel=kconf operation, but creates a netbsd.gdb file
550 1.106 wiz alongside of the kernel netbsd, which contains a full
551 1.106 wiz symbol table and can be used for debugging (for example
552 1.106 wiz with a cross-gdb built by MKCROSSGDB).
553 1.106 wiz
554 1.113 uebayasi kernels This command will build all kernels defined in port
555 1.113 uebayasi specific release build procedure.
556 1.111 uebayasi
557 1.112 uebayasi This command internally calls the kernel=kconf operation
558 1.112 uebayasi for each found kernel configuration file.
559 1.111 uebayasi
560 1.105 wiz modules This command will build kernel modules and install them
561 1.105 wiz into DESTDIR.
562 1.86 jnemeth
563 1.105 wiz releasekernel=kconf
564 1.70 dholland Install a gzip(1)ed copy of the kernel previously built by
565 1.105 wiz kernel=kconf into
566 1.105 wiz RELEASEDIR/RELEASEMACHINEDIR/binary/kernel, usually as
567 1.126 wiz netbsd-kconf.gz, although the "netbsd" prefix is determined
568 1.126 wiz from the "config" directives in kconf.
569 1.24 lukem
570 1.126 wiz sets Perform "make sets".
571 1.31 lukem
572 1.126 wiz sourcesets Perform "make sourcesets".
573 1.22 lukem
574 1.126 wiz syspkgs Perform "make syspkgs".
575 1.54 apb
576 1.126 wiz iso-image Perform "make iso-image".
577 1.56 apb
578 1.105 wiz iso-image-source
579 1.126 wiz Perform "make iso-image-source".
580 1.65 jnemeth
581 1.105 wiz install-image
582 1.126 wiz Perform "make install-image".
583 1.97 tsutsui
584 1.126 wiz live-image Perform "make live-image".
585 1.97 tsutsui
586 1.148 lukem list-arch Show a list of valid MACHINE and MACHINE_ARCH settings, the
587 1.148 lukem default MACHINE_ARCH for each MACHINE, and aliases for
588 1.109 apb MACHINE/MACHINE_ARCH pairs, and then exits. The -m or -a
589 1.109 apb options (or both) may be used to specify glob patterns that
590 1.109 apb will be used to narrow the list of results; for example,
591 1.131 leot "build.sh -m 'evb*' -a '*arm*' list-arch" will list all
592 1.109 apb known MACHINE/MACHINE_ARCH values in which either MACHINE
593 1.109 apb or ALIAS matches the pattern `evb*', and MACHINE_ARCH
594 1.109 apb matches the pattern `*arm*'.
595 1.109 apb
596 1.105 wiz The following command line options alter the behaviour of the build.sh
597 1.51 wiz operations described above:
598 1.1 tv
599 1.109 apb -a arch Set the value of MACHINE_ARCH to arch. See the -m option for
600 1.109 apb more information.
601 1.9 thorpej
602 1.105 wiz -B buildid
603 1.105 wiz Set the value of BUILDID to buildid. This will also append the
604 1.126 wiz build identifier to the name of the "make" wrapper script so
605 1.37 lukem that the resulting name is of the form
606 1.126 wiz "nbmake-MACHINE-BUILDID".
607 1.1 tv
608 1.105 wiz -C cdextras
609 1.105 wiz Append cdextras to the CDEXTRA variable, which is a space-
610 1.104 wiz separated list of files or directories that will be added to
611 1.126 wiz the CD-ROM image that may be create by the "iso-image" or
612 1.126 wiz "iso-image-source" operations. Files will be added to the root
613 1.126 wiz of the CD-ROM image, whereas directories will be copied
614 1.87 apb recursively. If relative paths are specified, they will be
615 1.87 apb converted to absolute paths before being used. Multiple paths
616 1.105 wiz may be specified via multiple -C options, or via a single
617 1.87 apb option whose argument contains multiple space-separated paths.
618 1.66 jnemeth
619 1.136 uwe -c compiler
620 1.136 uwe Select the compiler for the toolchain to build NetBSD and for
621 1.136 uwe inclusion in the NetBSD distribution. Supported choices:
622 1.136 uwe
623 1.136 uwe clang
624 1.136 uwe
625 1.136 uwe gcc [default]
626 1.136 uwe
627 1.136 uwe The compiler used to build the toolchain can be different; see
628 1.136 uwe HOST_CC and HOST_CXX.
629 1.136 uwe
630 1.105 wiz -D dest Set the value of DESTDIR to dest. If a relative path is
631 1.104 wiz specified, it will be converted to an absolute path before
632 1.104 wiz being used.
633 1.104 wiz
634 1.105 wiz -E Set `expert' mode. This overrides various sanity checks, and
635 1.105 wiz allows: DESTDIR does not have to be set to a non-root path for
636 1.105 wiz builds, and MKUNPRIVED=yes does not have to be set when
637 1.104 wiz building as a non-root user.
638 1.23 lukem
639 1.105 wiz Note: It is highly recommended that you know what you are doing
640 1.23 lukem when you use this option.
641 1.19 lukem
642 1.148 lukem -h Show a help message, and exit.
643 1.51 wiz
644 1.105 wiz -j njob Run up to njob make(1) subjobs in parallel; passed through to
645 1.74 apb make(1). If you see failures for reasons other than running
646 1.105 wiz out of memory while using build.sh with -j, please save
647 1.104 wiz complete build logs so the failures can be analyzed.
648 1.74 apb
649 1.105 wiz To achieve the fastest builds, -j values between (1 + the
650 1.104 wiz number of CPUs) and (2 * the number of CPUs) are recommended.
651 1.104 wiz Use lower values on machines with limited memory or I/O
652 1.104 wiz bandwidth.
653 1.104 wiz
654 1.126 wiz -M obj Set MAKEOBJDIRPREFIX to obj. Unsets MAKEOBJDIR. See "-O obj"
655 1.126 wiz for more information.
656 1.104 wiz
657 1.105 wiz For instance, if the source directory is /usr/src, a setting of
658 1.126 wiz "-M /usr/obj" will place build-time files under
659 1.105 wiz /usr/obj/usr/src/bin, /usr/obj/usr/src/lib,
660 1.105 wiz /usr/obj/usr/src/usr.bin, and so forth.
661 1.88 apb
662 1.73 apb If a relative path is specified, it will be converted to an
663 1.105 wiz absolute path before being used. build.sh imposes the
664 1.105 wiz restriction that the argument to the -M option must not begin
665 1.126 wiz with a "$" (dollar sign) character; otherwise it would be too
666 1.104 wiz difficult to determine whether the value is an absolute or a
667 1.104 wiz relative path. If the directory does not already exist,
668 1.105 wiz build.sh will create it.
669 1.18 lukem
670 1.109 apb -m mach Set the value of MACHINE to mach, unless the mach argument is
671 1.109 apb an alias that refers to a MACHINE/MACHINE_ARCH pair, in which
672 1.109 apb case both MACHINE and MACHINE_ARCH are set from the alias.
673 1.109 apb Such aliases are interpreted entirely by build.sh; they are not
674 1.109 apb used by any other part of the build system. The MACHINE_ARCH
675 1.109 apb setting implied by mach will override any value of MACHINE_ARCH
676 1.109 apb in the process environment, but will not override a value set
677 1.109 apb by the -a option. All cross builds require -m, but if unset on
678 1.109 apb a NetBSD host, the host's value of MACHINE will be detected and
679 1.109 apb used automatically.
680 1.109 apb
681 1.109 apb See the list-arch operation for a way to get a list of valid
682 1.109 apb MACHINE and MACHINE_ARCH settings.
683 1.1 tv
684 1.105 wiz -N noiselevel
685 1.126 wiz Set the "noisyness" level of the build, by setting MAKEVERBOSE
686 1.126 wiz to noiselevel.
687 1.45 lukem
688 1.105 wiz -n Show the commands that would be executed by build.sh, but do
689 1.126 wiz not make any changes. This is similar in concept to "make -n".
690 1.1 tv
691 1.105 wiz -O obj Create an appropriate transform macro for MAKEOBJDIR that will
692 1.105 wiz place the built object files under obj. Unsets
693 1.105 wiz MAKEOBJDIRPREFIX.
694 1.88 apb
695 1.126 wiz For instance, a setting of "-O /usr/obj" will place build-time
696 1.126 wiz files under /usr/obj/bin, /usr/obj/lib, /usr/obj/usr.bin, and
697 1.126 wiz so forth.
698 1.88 apb
699 1.88 apb If a relative path is specified, it will be converted to an
700 1.105 wiz absolute path before being used. build.sh imposes the
701 1.105 wiz restriction that the argument to the -O option must not contain
702 1.126 wiz a "$" (dollar sign) character. If the directory does not
703 1.105 wiz already exist, build.sh will create it.
704 1.73 apb
705 1.105 wiz In normal use, exactly one of the -M or -O options should be
706 1.105 wiz specified. If neither -M nor -O is specified, then a default
707 1.78 apb object directory will be chosen according to rules in
708 1.78 apb <bsd.obj.mk>. Relying on this default is not recommended
709 1.78 apb because it is determined by complex rules that are influenced
710 1.78 apb by the values of several variables and by the location of the
711 1.78 apb source directory.
712 1.18 lukem
713 1.151 lukem Note: Placing the obj directory location outside of the default
714 1.151 lukem source tree hierarchy makes it easier to manually clear out old
715 1.151 lukem files in the event the "make cleandir" operation is unable to
716 1.151 lukem do so. (See CAVEATS below.)
717 1.151 lukem
718 1.151 lukem Note: The use of one of -M or -O is the only means of building
719 1.151 lukem multiple machine architecture userlands from the same source
720 1.151 lukem tree without cleaning between builds (in which case, one would
721 1.151 lukem specify distinct obj locations for each).
722 1.109 apb
723 1.126 wiz -o Set the value of MKOBJDIRS to "no". Otherwise, it will be
724 1.126 wiz automatically set to "yes". This default is opposite to the
725 1.105 wiz behaviour when not using build.sh.
726 1.1 tv
727 1.148 lukem -P Set the value of MKREPRO and MKREPRO_TIMESTAMP to the latest
728 1.148 lukem source CVS timestamp for reproducible builds.
729 1.148 lukem
730 1.105 wiz -R rel Set the value of RELEASEDIR to rel. If a relative path is
731 1.59 apb specified, it will be converted to an absolute path before
732 1.59 apb being used.
733 1.18 lukem
734 1.105 wiz -r Remove the contents of DESTDIR and TOOLDIR before building
735 1.1 tv (provides a clean starting point). This will skip deleting
736 1.105 wiz DESTDIR if building on a native system to the root directory.
737 1.1 tv
738 1.105 wiz -S seed Change the value of BUILDSEED to seed. This should rarely be
739 1.72 perry necessary.
740 1.72 perry
741 1.105 wiz -T tools Set the value of TOOLDIR to tools. If a relative path is
742 1.104 wiz specified, it will be converted to an absolute path before
743 1.126 wiz being used. If set, the bootstrap "make" will only be rebuilt
744 1.126 wiz if the source files for make(1) have changed.
745 1.104 wiz
746 1.105 wiz -U Set MKUNPRIVED=yes.
747 1.104 wiz
748 1.105 wiz -u Set MKUPDATE=yes.
749 1.104 wiz
750 1.105 wiz -V var=[value]
751 1.105 wiz Set the environment variable var to an optional value. This is
752 1.105 wiz propagated to the nbmake wrapper.
753 1.105 wiz
754 1.105 wiz -w wrapper
755 1.105 wiz Create the nbmake wrapper script (see below) in a custom
756 1.105 wiz location, specified by wrapper. This allows, for instance, to
757 1.151 lukem place the wrapper in PATH automatically.
758 1.151 lukem
759 1.151 lukem Note: wrapper is the full name of the file, not just a
760 1.151 lukem directory name. If a relative path is specified, it will be
761 1.151 lukem converted to an absolute path before being used.
762 1.1 tv
763 1.105 wiz -X x11src
764 1.105 wiz Set the value of X11SRCDIR to x11src. If a relative path is
765 1.59 apb specified, it will be converted to an absolute path before
766 1.59 apb being used.
767 1.49 lukem
768 1.105 wiz -x Set MKX11=yes.
769 1.49 lukem
770 1.105 wiz -Z var Unset ("zap") the environment variable var. This is propagated
771 1.105 wiz to the nbmake wrapper.
772 1.40 lukem
773 1.148 lukem -? Show a help message, and exit.
774 1.148 lukem
775 1.105 wiz The "nbmake-MACHINE" wrapper script
776 1.105 wiz If using the build.sh script to build NetBSD, a nbmake-MACHINE script
777 1.105 wiz will be created in TOOLDIR/bin upon the first build to assist in building
778 1.1 tv subtrees on a cross-compile host.
779 1.1 tv
780 1.105 wiz nbmake-MACHINE can be invoked in lieu of make(1), and will instead call
781 1.126 wiz the up-to-date version of "nbmake" installed into TOOLDIR/bin with
782 1.105 wiz several key variables pre-set, including MACHINE, MACHINE_ARCH, and
783 1.105 wiz TOOLDIR. nbmake-MACHINE will also set variables specified with -V, and
784 1.105 wiz unset variables specified with -Z.
785 1.40 lukem
786 1.105 wiz This script can be symlinked into a directory listed in PATH, or called
787 1.1 tv with an absolute path.
788 1.1 tv
789 1.105 wiz EXAMPLES
790 1.151 lukem 1. % ./build.sh [OPTIONS] tools kernel=GENERIC
791 1.28 lukem
792 1.28 lukem Build a new toolchain, and use the new toolchain to configure and
793 1.28 lukem build a new GENERIC kernel.
794 1.28 lukem
795 1.151 lukem 2. % ./build.sh [OPTIONS] -U distribution
796 1.28 lukem
797 1.105 wiz Using unprivileged mode, build a complete distribution to a DESTDIR
798 1.148 lukem directory that build.sh selects (and will show).
799 1.28 lukem
800 1.151 lukem 3. # ./build.sh [OPTIONS] -U install=/
801 1.28 lukem
802 1.105 wiz As root, install to / the distribution that was built by example 2.
803 1.105 wiz Even though this is run as root, -U is required so that the
804 1.105 wiz permissions stored in DESTDIR/METALOG are correctly applied to the
805 1.105 wiz files as they're copied to /.
806 1.28 lukem
807 1.151 lukem 4. % ./build.sh [OPTIONS] -U -u release
808 1.28 lukem
809 1.105 wiz Using unprivileged mode, build a complete release to DESTDIR and
810 1.148 lukem RELEASEDIR directories that build.sh selects (and will show).
811 1.126 wiz MKUPDATE=yes (-u) is set to prevent the "make cleandir", so that if
812 1.126 wiz this is run after example 2, it doesn't need to redo that portion of
813 1.126 wiz the release build.
814 1.16 lukem
815 1.105 wiz SEE ALSO
816 1.155 lukem config(1), ctags(1), ex(1), gzip(1), lint(1), make(1), mandoc(1),
817 1.155 lukem mkisofs(1), nroff(1), vi(1), mk.conf(5), atf(7), hier(7), mdoc(7),
818 1.155 lukem release(7), etcupdate(8), installboot(8), mount(8), postinstall(8),
819 1.151 lukem sysinst(8), pkgsrc/sysutils/cdrtools
820 1.1 tv
821 1.155 lukem Note: The NetBSD manual pages are also available at
822 1.155 lukem https://man.netbsd.org
823 1.155 lukem
824 1.105 wiz HISTORY
825 1.105 wiz The build.sh based build scheme was introduced for NetBSD 1.6 as
826 1.105 wiz USE_NEW_TOOLCHAIN, and re-worked to TOOLCHAIN_MISSING after that.
827 1.1 tv
828 1.109 apb CAVEATS
829 1.109 apb After significant updates to third-party components in the source tree,
830 1.126 wiz the "make cleandir" operation may be insufficient to clean out old files
831 1.126 wiz in object directories. Instead, one may have to manually remove the
832 1.126 wiz files. Consult the UPDATING file for notices concerning this.
833 1.109 apb
834 1.157 lukem NetBSD July 18, 2023 NetBSD
835