BUILDING revision 1.159 1 1.104 wiz BUILDING(8) System Manager's Manual BUILDING(8)
2 1.1 tv
3 1.105 wiz NAME
4 1.150 lukem BUILDING - Procedure for building NetBSD from source code
5 1.1 tv
6 1.105 wiz REQUIREMENTS
7 1.1 tv NetBSD is designed to be buildable on most POSIX-compliant host systems.
8 1.105 wiz The basic build procedure is the same whether compiling natively (on the
9 1.105 wiz same NetBSD architecture) or cross compiling (on another architecture or
10 1.1 tv OS).
11 1.1 tv
12 1.157 lukem This source tree contains the build.sh shell script which supports both
13 1.157 lukem native and cross builds of NetBSD.
14 1.157 lukem
15 1.126 wiz This source tree contains a special subtree, "tools", which uses the host
16 1.126 wiz system to create a build toolchain for the target architecture. The host
17 1.126 wiz system must have at least C and C++ compilers in order to create the
18 1.157 lukem toolchain (make(1) is not required); all other tools (including make(1)
19 1.157 lukem as nbmake) are created as part of the NetBSD build process. (See the
20 1.157 lukem Environment variables section below if you need to override or manually
21 1.157 lukem select your compilers.)
22 1.1 tv
23 1.155 lukem Note: Within this document, cross-references to manual pages are to the
24 1.155 lukem NetBSD manual pages, not the host system manual pages. The mdoc(7)
25 1.155 lukem source to the NetBSD manual pages can be found within the source tree,
26 1.155 lukem and these and can be formatted with mandoc(1) or nroff(1) if those are
27 1.155 lukem available on the host system. The NetBSD manual pages are also available
28 1.155 lukem at https://man.netbsd.org
29 1.155 lukem
30 1.105 wiz FILES
31 1.105 wiz Source tree layout
32 1.151 lukem BUILDING This document (in plaintext). Generated from
33 1.151 lukem doc/BUILDING.mdoc.
34 1.48 jmc
35 1.105 wiz Makefile The main Makefile for NetBSD; should only be run for
36 1.44 lukem native builds with an appropriately up-to-date version of
37 1.142 andvar NetBSD make(1). Intended for expert use with knowledge of
38 1.129 sevan its shortcomings, it has been superseded by the build.sh
39 1.129 sevan shell script as the recommended means for building NetBSD.
40 1.1 tv
41 1.105 wiz UPDATING Special notes for updating from an earlier revision of
42 1.1 tv NetBSD. It is important to read this file before every
43 1.1 tv build of an updated source tree.
44 1.1 tv
45 1.105 wiz build.sh Bourne-compatible shell script used for building the host
46 1.1 tv build tools and the NetBSD system from scratch. Can be
47 1.1 tv used for both native and cross builds, and should be used
48 1.129 sevan instead of make(1) as it performs additional checks to
49 1.129 sevan prevent common issues going undetected, such as building
50 1.129 sevan with an outdated version of make(1).
51 1.1 tv
52 1.105 wiz crypto/dist/, dist/, gnu/dist/
53 1.104 wiz Sources imported verbatim from third parties, without
54 1.104 wiz mangling the existing build structure. Other source trees
55 1.126 wiz in bin through usr.sbin use the NetBSD make(1) "reachover"
56 1.126 wiz Makefile semantics when building these programs for a
57 1.126 wiz native host.
58 1.104 wiz
59 1.151 lukem distrib/, etc/
60 1.151 lukem Sources for items used when making a full release
61 1.151 lukem snapshot, such as files installed in DESTDIR/etc on the
62 1.151 lukem destination system, boot media, and release notes.
63 1.151 lukem
64 1.151 lukem doc/BUILDING.mdoc
65 1.155 lukem The source to this document, in mdoc(7) format. Used to
66 1.155 lukem generate BUILDING.
67 1.151 lukem
68 1.158 lukem external/, sys/external/
69 1.106 wiz Sources and build infrastructure for components imported
70 1.106 wiz (mostly) unchanged from upstream maintainers, sorted by
71 1.106 wiz applicable license. This is (slowly) replacing the
72 1.106 wiz crypto/dist, dist, and gnu/dist directories.
73 1.106 wiz
74 1.151 lukem external/mit/xorg/
75 1.151 lukem "Reachover" build structure for modular Xorg; the source
76 1.151 lukem is in X11SRCDIR.
77 1.1 tv
78 1.157 lukem mk.conf Optional source tree specific mk.conf(5), used (if
79 1.157 lukem present) instead of /etc/mk.conf unless MAKECONF is
80 1.157 lukem defined.
81 1.157 lukem
82 1.157 lukem Note: Not part of the NetBSD source repository.
83 1.157 lukem
84 1.151 lukem regress/, tests/
85 1.70 dholland Regression test harness. Can be cross-compiled, but only
86 1.105 wiz run natively. tests/ uses the atf(7) test framework;
87 1.105 wiz regress/ contains older tests that have not yet been
88 1.70 dholland migrated to atf(7).
89 1.1 tv
90 1.105 wiz sys/ NetBSD kernel sources.
91 1.1 tv
92 1.126 wiz tools/ "Reachover" build structure for the host build tools.
93 1.104 wiz This has a special method of determining out-of-date
94 1.104 wiz status.
95 1.1 tv
96 1.151 lukem tools/compat/README
97 1.151 lukem Special notes for cross-hosting a NetBSD build on non-
98 1.151 lukem NetBSD platforms.
99 1.151 lukem
100 1.151 lukem Other directories including bin/ ... usr.sbin/
101 1.1 tv Sources to the NetBSD userland (non-kernel) programs. If
102 1.1 tv any of these directories are missing, they will be skipped
103 1.1 tv during the build.
104 1.1 tv
105 1.105 wiz Build tree layout
106 1.156 lukem The NetBSD build tree is described in hier(7) (whose mdoc(7) source is in
107 1.155 lukem share/man/man7/hier.7), and the release layout is described in release(7)
108 1.156 lukem (whose mdoc(7) source is in share/man/man7/release.7).
109 1.1 tv
110 1.105 wiz CONFIGURATION
111 1.105 wiz Environment variables
112 1.8 lukem Several environment variables control the behaviour of NetBSD builds.
113 1.8 lukem
114 1.152 lukem HOST_CC Path name to C compiler used to create the toolchain.
115 1.58 apb
116 1.155 lukem Default: "cc".
117 1.155 lukem
118 1.152 lukem HOST_CFLAGS Flags passed to the host C compiler.
119 1.53 reed
120 1.155 lukem Default: "-O".
121 1.155 lukem
122 1.155 lukem HOST_CPPFLAGS Flags passed to the host C/C++ pre-processor.
123 1.155 lukem
124 1.155 lukem Default: Unset.
125 1.155 lukem
126 1.152 lukem HOST_CXX Path name to C++ compiler used to create the toolchain.
127 1.137 uwe
128 1.155 lukem Default: Unset, but defaults to "c++" where required.
129 1.155 lukem
130 1.152 lukem HOST_CXXFLAGS Flags passed to the host C++ compiler.
131 1.53 reed
132 1.155 lukem Default: Unset.
133 1.155 lukem
134 1.152 lukem HOST_SH Path name to a shell available on the host system and
135 1.152 lukem suitable for use during the build. The NetBSD build
136 1.152 lukem system requires a modern Bourne-like shell with POSIX-
137 1.152 lukem compliant features, and also requires support for the
138 1.152 lukem "local" keyword to declare local variables in shell
139 1.152 lukem functions (which is a widely-implemented but non-
140 1.152 lukem standardised feature).
141 1.152 lukem
142 1.152 lukem Depending on the host system, a suitable shell may be
143 1.152 lukem /bin/sh, /usr/xpg4/bin/sh, /bin/ksh (provided it is a
144 1.152 lukem variant of ksh that supports the "local" keyword, such as
145 1.152 lukem ksh88, but not ksh93), or /usr/local/bin/bash.
146 1.152 lukem
147 1.152 lukem Most parts of the build require HOST_SH to be an absolute
148 1.152 lukem path; however, build.sh allows it to be a simple command
149 1.152 lukem name, which will be converted to an absolute path by
150 1.152 lukem searching the PATH.
151 1.137 uwe
152 1.155 lukem Default: "sh".
153 1.155 lukem
154 1.147 lukem INSTALLBOOT_UBOOT_PATHS
155 1.152 lukem A colon-separated list of search paths used by
156 1.152 lukem installboot(8) to find U-Boot packages.
157 1.147 lukem
158 1.155 lukem Default: Unset.
159 1.155 lukem
160 1.152 lukem MACHINE Machine type, e.g., "macppc".
161 1.8 lukem
162 1.155 lukem Default: Unset.
163 1.155 lukem
164 1.152 lukem MACHINE_ARCH Machine architecture, e.g., "powerpc".
165 1.8 lukem
166 1.155 lukem Default: Unset.
167 1.155 lukem
168 1.152 lukem MAKE Path name to invoke make(1) as.
169 1.8 lukem
170 1.155 lukem Default: "make".
171 1.155 lukem
172 1.152 lukem MAKECONF The name of the make(1) configuration file. See "make"
173 1.152 lukem variables and mk.conf(5).
174 1.151 lukem
175 1.152 lukem Note: Only settable in the process environment.
176 1.151 lukem
177 1.157 lukem Default: "/etc/mk.conf", although build.sh will set the
178 1.157 lukem default to the full path to mk.conf if the latter is
179 1.157 lukem present in the same directory as build.sh.
180 1.151 lukem
181 1.152 lukem MAKEFLAGS Flags to invoke make(1) with.
182 1.151 lukem
183 1.152 lukem Note: build.sh ignores the value of MAKEFLAGS passed in
184 1.152 lukem the environment, but allows MAKEFLAGS to be set via the
185 1.152 lukem -V option.
186 1.151 lukem
187 1.155 lukem Default: "-X" on systems with a small ARG_MAX (Cygwin,
188 1.155 lukem Darwin, FreeBSD); otherwise unset.
189 1.155 lukem
190 1.152 lukem MAKEOBJDIR Directory to use as the .OBJDIR for the current
191 1.152 lukem directory. The value is subjected to variable expansion
192 1.152 lukem by make(1). Typical usage is to set this variable to a
193 1.152 lukem value involving the use of `${.CURDIR:S...}' or
194 1.152 lukem `${.CURDIR:C...}', to derive the value of .OBJDIR from
195 1.152 lukem the value of .CURDIR. Used only if MAKEOBJDIRPREFIX is
196 1.152 lukem not defined.
197 1.151 lukem
198 1.152 lukem Note: MAKEOBJDIR can be provided only in the environment
199 1.152 lukem or via the -O flag of build.sh; it cannot usefully be set
200 1.152 lukem inside a Makefile, including in mk.conf(5) or MAKECONF.
201 1.151 lukem
202 1.155 lukem Default: Unset.
203 1.155 lukem
204 1.151 lukem MAKEOBJDIRPREFIX
205 1.152 lukem Top level directory of the object directory tree. The
206 1.152 lukem value is subjected to variable expansion by make(1).
207 1.152 lukem build.sh will create the ${MAKEOBJDIRPREFIX} directory if
208 1.152 lukem necessary, but if make(1) is used without build.sh, then
209 1.152 lukem rules in <bsd.obj.mk> will abort the build if the
210 1.152 lukem ${MAKEOBJDIRPREFIX} directory does not exist. If the
211 1.152 lukem value is defined and valid, then
212 1.152 lukem ${MAKEOBJDIRPREFIX}/${.CURDIR} is used as the .OBJDIR for
213 1.152 lukem the current directory. The current directory may be read
214 1.152 lukem only.
215 1.152 lukem
216 1.152 lukem Note: MAKEOBJDIRPREFIX can be provided only in the
217 1.152 lukem environment or via the -M flag of build.sh; it cannot
218 1.152 lukem usefully be set inside a Makefile, including in
219 1.152 lukem mk.conf(5) or MAKECONF.
220 1.152 lukem
221 1.155 lukem Default: Unset.
222 1.155 lukem
223 1.152 lukem TMPDIR Top-level directory to store temporary directories used
224 1.152 lukem by build.sh before paths to other directories such as
225 1.152 lukem .OBJDIR can be determined.
226 1.149 lukem
227 1.152 lukem Note: Must support execution of binaries. I.e., without
228 1.152 lukem mount(8)'s -o noexec option.
229 1.149 lukem
230 1.152 lukem Default: "/tmp".
231 1.149 lukem
232 1.105 wiz "make" variables
233 1.156 lukem Variables that control the behavior of NetBSD builds are documented in
234 1.156 lukem mk.conf(5) (whose mdoc(7) source is in share/man/man5/mk.conf.5).
235 1.45 lukem
236 1.155 lukem Unless otherwise specified, these variables may be set in either the
237 1.155 lukem process environment or the make(1) configuration file mk.conf(5)
238 1.155 lukem specified by MAKECONF.
239 1.49 lukem
240 1.159 lukem Note: Variables set in the environment, either directly or via build.sh
241 1.159 lukem options to set specific values in the nbmake-MACHINE wrapper script do
242 1.159 lukem not override variables set in the mk.conf(5) file. To allow variables in
243 1.159 lukem mk.conf(5) to be overridden by the environment or build.sh options,
244 1.159 lukem define the variables using the "?=" make(1) variable assignment operator.
245 1.159 lukem For example,
246 1.159 lukem
247 1.159 lukem MAKEVERBOSE?=1
248 1.159 lukem
249 1.105 wiz BUILDING
250 1.105 wiz "make" command line options
251 1.52 jmc This is not a summary of all the options available to make(1); only the
252 1.52 jmc options used most frequently with NetBSD builds are listed here.
253 1.1 tv
254 1.105 wiz -j njob Run up to njob make(1) subjobs in parallel. Makefiles should
255 1.52 jmc use .WAIT or have explicit dependencies as necessary to
256 1.74 apb enforce build ordering.
257 1.10 sommerfe
258 1.105 wiz -m dir Specify the default directory for searching for system
259 1.104 wiz Makefile segments, mainly the <bsd.*.mk> files. When building
260 1.104 wiz any full NetBSD source tree, this should be set to the
261 1.126 wiz "share/mk" directory in the source tree. This is set
262 1.104 wiz automatically when building from the top level, or when using
263 1.105 wiz build.sh.
264 1.1 tv
265 1.148 lukem -n Show the commands that would have been executed, but do not
266 1.1 tv actually execute them. This will still cause recursion to
267 1.1 tv take place.
268 1.1 tv
269 1.148 lukem -V var Show make(1)'s idea of the value of var. Does not build any
270 1.1 tv targets.
271 1.1 tv
272 1.105 wiz var=value Set the variable var to value, overriding any setting
273 1.105 wiz specified by the process environment, the MAKECONF
274 1.104 wiz configuration file, or the system Makefile segments.
275 1.1 tv
276 1.105 wiz "make" targets
277 1.1 tv These default targets may be built by running make(1) in any subtree of
278 1.1 tv the NetBSD source code. It is recommended that none of these be used
279 1.126 wiz from the top level Makefile; as a specific exception, "make obj" and
280 1.126 wiz "make cleandir" are useful in that context.
281 1.1 tv
282 1.105 wiz all Build programs, libraries, and preformatted documentation.
283 1.1 tv
284 1.105 wiz clean Remove program and library object code files.
285 1.1 tv
286 1.105 wiz cleandir Same as clean, but also remove preformatted documentation,
287 1.126 wiz dependency files generated by "make depend", and any other
288 1.49 lukem files known to be created at build time.
289 1.1 tv
290 1.105 wiz depend Create dependency files (.depend) containing more detailed
291 1.44 lukem information about the dependencies of source code on header
292 1.1 tv files. Allows programs to be recompiled automatically when a
293 1.1 tv dependency changes.
294 1.1 tv
295 1.126 wiz dependall Does a "make depend" immediately followed by a "make all".
296 1.10 sommerfe This improves cache locality of the build since both passes
297 1.10 sommerfe read the source files in their entirety.
298 1.1 tv
299 1.105 wiz distclean Synonym for cleandir.
300 1.49 lukem
301 1.105 wiz includes Build and install system header files. Typically needed
302 1.44 lukem before any system libraries or programs can be built.
303 1.1 tv
304 1.105 wiz install Install programs, libraries, and documentation into DESTDIR.
305 1.105 wiz Few files will be installed to DESTDIR/dev, DESTDIR/etc,
306 1.105 wiz DESTDIR/root or DESTDIR/var in order to prevent user supplied
307 1.27 lukem configuration data from being overwritten.
308 1.1 tv
309 1.105 wiz lint Run lint(1) against the C source code, where appropriate, and
310 1.1 tv generate system-installed lint libraries.
311 1.1 tv
312 1.105 wiz obj Create object directories to be used for built files, instead
313 1.1 tv of building directly in the source tree.
314 1.1 tv
315 1.105 wiz tags Create ctags(1) searchable function lists usable by the ex(1)
316 1.1 tv and vi(1) text editors.
317 1.1 tv
318 1.105 wiz "make" targets for the top level
319 1.1 tv Additional make(1) targets are usable specifically from the top source
320 1.1 tv level to facilitate building the entire NetBSD source tree.
321 1.1 tv
322 1.105 wiz build Build the entire NetBSD system (except the kernel). This
323 1.68 reed orders portions of the source tree such that prerequisites
324 1.68 reed will be built in the proper order.
325 1.20 lukem
326 1.126 wiz distribution Do a "make build", and then install a full distribution
327 1.105 wiz (which does not include a kernel) into DESTDIR, including
328 1.105 wiz files in DESTDIR/dev, DESTDIR/etc, DESTDIR/root and
329 1.105 wiz DESTDIR/var.
330 1.20 lukem
331 1.126 wiz buildworld As per "make distribution", except that it ensures that
332 1.105 wiz DESTDIR is not the root directory.
333 1.20 lukem
334 1.105 wiz installworld Install the distribution from DESTDIR to INSTALLWORLDDIR,
335 1.70 dholland which defaults to the root directory. Ensures that
336 1.105 wiz INSTALLWORLDDIR is not the root directory if cross
337 1.104 wiz compiling.
338 1.20 lukem
339 1.105 wiz The INSTALLSETS environment variable may be set to a space-
340 1.95 apb separated list of distribution sets to be installed. By
341 1.126 wiz default, all sets except "etc" and "xetc" are installed, so
342 1.126 wiz most files in INSTALLWORLDDIR/etc will not be installed or
343 1.126 wiz modified.
344 1.61 apb
345 1.105 wiz Note: Before performing this operation with
346 1.105 wiz INSTALLWORLDDIR=/, it is highly recommended that you
347 1.61 apb upgrade your kernel and reboot. After performing this
348 1.61 apb operation, it is recommended that you use etcupdate(8) to
349 1.105 wiz update files in INSTALLWORLDDIR/etc, and postinstall(8) to
350 1.102 abs check for or fix inconsistencies.
351 1.20 lukem
352 1.105 wiz sets Create distribution sets from DESTDIR into
353 1.105 wiz RELEASEDIR/RELEASEMACHINEDIR/binary/sets. Should be run
354 1.126 wiz after "make distribution", as "make build" alone does not
355 1.126 wiz install all of the required files.
356 1.25 lukem
357 1.105 wiz sourcesets Create source sets of the source tree into
358 1.105 wiz RELEASEDIR/source/sets.
359 1.31 lukem
360 1.105 wiz syspkgs Create syspkgs from DESTDIR into
361 1.105 wiz RELEASEDIR/RELEASEMACHINEDIR/binary/syspkgs. Should be run
362 1.126 wiz after "make distribution", as "make build" alone does not
363 1.126 wiz install all of the required files.
364 1.54 apb
365 1.126 wiz release Do a "make distribution", build kernels, distribution
366 1.126 wiz media, and install sets (this as per "make sets"), and then
367 1.126 wiz package the system into a standard release layout as
368 1.105 wiz described by release(7). This requires that RELEASEDIR be
369 1.25 lukem set (see above).
370 1.1 tv
371 1.105 wiz iso-image Create a NetBSD installation CD-ROM image in the
372 1.114 snj RELEASEDIR/images directory. The CD-ROM file system will
373 1.114 snj have a layout as described in release(7).
374 1.62 apb
375 1.62 apb For most machine types, the CD-ROM will be bootable, and
376 1.104 wiz will automatically run the sysinst(8) menu-based
377 1.104 wiz installation program, which can be used to install or
378 1.104 wiz upgrade a NetBSD system. Bootable CD-ROMs also contain
379 1.104 wiz tools that may be useful in repairing a damaged NetBSD
380 1.104 wiz installation.
381 1.62 apb
382 1.126 wiz Before "make iso-image" is attempted, RELEASEDIR must be
383 1.126 wiz populated by "make release" or equivalent.
384 1.62 apb
385 1.151 lukem Note: Other, smaller, CD-ROM images may be created in the
386 1.151 lukem RELEASEDIR/RELEASEMACHINEDIR/installation/cdrom directory
387 1.151 lukem by "make release". These smaller images usually contain
388 1.151 lukem the same tools as the larger images in RELEASEDIR/images,
389 1.151 lukem but do not contain additional content such as the
390 1.151 lukem distribution sets.
391 1.62 apb
392 1.151 lukem Note: The mac68k port still uses an older method of
393 1.104 wiz creating CD-ROM images. This requires the mkisofs(1)
394 1.104 wiz utility, which is not part of NetBSD, but which can be
395 1.105 wiz installed from pkgsrc/sysutils/cdrtools.
396 1.56 apb
397 1.105 wiz iso-image-source
398 1.64 jnemeth Create a NetBSD installation CD-ROM image in the
399 1.114 snj RELEASEDIR/images directory. The CD-ROM file system will
400 1.114 snj have a layout as described in release(7). It will have top
401 1.64 jnemeth level directories for the machine type and source.
402 1.64 jnemeth
403 1.64 jnemeth For most machine types, the CD-ROM will be bootable, and
404 1.104 wiz will automatically run the sysinst(8) menu-based
405 1.104 wiz installation program, which can be used to install or
406 1.104 wiz upgrade a NetBSD system. Bootable CD-ROMs also contain
407 1.104 wiz tools that may be useful in repairing a damaged NetBSD
408 1.104 wiz installation.
409 1.64 jnemeth
410 1.126 wiz Before "make iso-image-source" is attempted, RELEASEDIR
411 1.126 wiz must be populated by "make sourcesets release" or
412 1.104 wiz equivalent.
413 1.64 jnemeth
414 1.151 lukem Note: Other, smaller, CD-ROM images may be created in the
415 1.151 lukem RELEASEDIR/RELEASEMACHINEDIR/installation/cdrom directory
416 1.151 lukem by "make release". These smaller images usually contain
417 1.151 lukem the same tools as the larger images in RELEASEDIR/images,
418 1.151 lukem but do not contain additional content such as the
419 1.151 lukem distribution sets.
420 1.64 jnemeth
421 1.151 lukem Note: The mac68k port still uses an older method of
422 1.104 wiz creating CD-ROM images. This requires the mkisofs(1)
423 1.104 wiz utility, which is not part of NetBSD, but which can be
424 1.105 wiz installed from pkgsrc/sysutils/cdrtools.
425 1.64 jnemeth
426 1.105 wiz install-image
427 1.98 apb Create a bootable NetBSD installation disk image in the
428 1.127 plunky RELEASEDIR/images directory. The installation disk image
429 1.127 plunky is suitable for copying to bootable USB flash memory
430 1.127 plunky sticks, etc., for machines which are able to boot from such
431 1.127 plunky devices. The file system in the bootable disk image will
432 1.127 plunky have a layout as described in release(7).
433 1.97 tsutsui
434 1.97 tsutsui The installation image is bootable, and will automatically
435 1.97 tsutsui run the sysinst(8) menu-based installation program, which
436 1.98 apb can be used to install or upgrade a NetBSD system. The
437 1.98 apb image also contains tools that may be useful in repairing a
438 1.98 apb damaged NetBSD installation.
439 1.97 tsutsui
440 1.126 wiz Before "make install-image" is attempted, RELEASEDIR must
441 1.126 wiz be populated by "make release" or equivalent. The build
442 1.126 wiz must have been performed with MKUNPRIVED=yes because "make
443 1.126 wiz install-image" relies on information in DESTDIR/METALOG.
444 1.97 tsutsui
445 1.120 pgoyette live-image Create NetBSD live images in the RELEASEDIR/images
446 1.104 wiz directory. The live image contains all necessary files to
447 1.104 wiz boot NetBSD up to multi-user mode, including all files
448 1.104 wiz which should be extracted during installation, NetBSD
449 1.104 wiz disklabel, bootloaders, etc.
450 1.104 wiz
451 1.104 wiz The live image is suitable for use as a disk image in
452 1.104 wiz virtual machine environments such as QEMU, and also useful
453 1.104 wiz to boot NetBSD from a USB flash memory stick on a real
454 1.98 apb machine, without the need for installation.
455 1.97 tsutsui
456 1.126 wiz Before "make live-image" is attempted, RELEASEDIR must be
457 1.126 wiz populated by "make release" or equivalent. The build must
458 1.126 wiz have been performed with MKUNPRIVED=yes because "make
459 1.126 wiz install-image" relies on information in DESTDIR/METALOG.
460 1.97 tsutsui
461 1.105 wiz regression-tests
462 1.20 lukem Can only be run after building the regression tests in the
463 1.126 wiz directory "regress". Runs those compiled regression tests
464 1.151 lukem on the local host.
465 1.151 lukem
466 1.151 lukem Note: Most tests are now managed instead using atf(7); this
467 1.151 lukem target should probably run those as well but currently does
468 1.151 lukem not.
469 1.1 tv
470 1.105 wiz The "build.sh" script
471 1.96 apb This script file is a shell script designed to build the entire NetBSD
472 1.104 wiz system on any host with a suitable modern shell and some common
473 1.105 wiz utilities. The required shell features are described under the HOST_SH
474 1.104 wiz variable.
475 1.96 apb
476 1.96 apb If a host system's default shell does support the required features, then
477 1.96 apb we suggest that you explicitly specify a suitable shell using a command
478 1.96 apb like
479 1.96 apb
480 1.105 wiz /path/to/suitable/shell build.sh [options]
481 1.96 apb
482 1.105 wiz The above command will usually enable build.sh to automatically set
483 1.105 wiz HOST_SH=/path/to/suitable/shell, but if that fails, then the following
484 1.96 apb set of commands may be used instead:
485 1.96 apb
486 1.105 wiz HOST_SH=/path/to/suitable/shell
487 1.96 apb export HOST_SH
488 1.105 wiz ${HOST_SH} build.sh [options]
489 1.96 apb
490 1.105 wiz If build.sh detects that it is being executed under an unsuitable shell,
491 1.148 lukem it attempts to exec a suitable shell instead, or shows an error message.
492 1.105 wiz If HOST_SH is not set explicitly, then build.sh sets a default using
493 1.96 apb heuristics dependent on the host platform, or from the shell under which
494 1.105 wiz build.sh is executed (if that can be determined), or using the first copy
495 1.105 wiz of sh found in PATH.
496 1.1 tv
497 1.1 tv All cross-compile builds, and most native builds, of the entire system
498 1.126 wiz should make use of build.sh rather than just running "make". This way,
499 1.104 wiz the make(1) program will be bootstrapped properly, in case the host
500 1.126 wiz system has an older or incompatible "make" program.
501 1.1 tv
502 1.105 wiz When compiling the entire system via build.sh, many make(1) variables are
503 1.1 tv set for you in order to help encapsulate the build process. In the list
504 1.105 wiz of options below, variables that are automatically set by build.sh are
505 1.1 tv noted where applicable.
506 1.1 tv
507 1.105 wiz The following operations are supported by build.sh:
508 1.22 lukem
509 1.126 wiz build Build the system as per "make build". Before the main part
510 1.126 wiz of the build commences, this command runs the obj operation
511 1.126 wiz (unless the -o option is given), "make cleandir" (unless
512 1.126 wiz the -u option is given), and the tools operation.
513 1.22 lukem
514 1.126 wiz distribution Build a full distribution as per "make distribution". This
515 1.126 wiz command first runs the build operation.
516 1.22 lukem
517 1.126 wiz release Build a full release as per "make release". This command
518 1.105 wiz first runs the distribution operation.
519 1.22 lukem
520 1.148 lukem help Show a help message, and exit.
521 1.148 lukem
522 1.159 lukem makewrapper Create the nbmake-MACHINE wrapper script. This operation
523 1.159 lukem is automatically performed for any of the other operations.
524 1.23 lukem
525 1.126 wiz cleandir Perform "make cleandir".
526 1.75 lukem
527 1.126 wiz obj Perform "make obj".
528 1.22 lukem
529 1.105 wiz tools Build and install the host tools from src/tools. This
530 1.126 wiz command will first run "make obj" and "make cleandir" in
531 1.126 wiz the tools subdirectory unless the -o or -u options
532 1.104 wiz (respectively) are given.
533 1.22 lukem
534 1.126 wiz install=idir Install the contents of DESTDIR to idir, using "make
535 1.151 lukem installworld".
536 1.151 lukem
537 1.151 lukem Note: Files that are part of the "etc" or "xetc" sets will
538 1.151 lukem not be installed, unless overridden by the INSTALLSETS
539 1.151 lukem environment variable.
540 1.35 lukem
541 1.105 wiz kernel=kconf Build a new kernel. The kconf argument is the name of a
542 1.105 wiz configuration file suitable for use by config(1). If kconf
543 1.22 lukem does not contain any `/' characters, the configuration file
544 1.105 wiz is expected to be found in the KERNCONFDIR directory, which
545 1.105 wiz is typically sys/arch/MACHINE/conf. The new kernel will be
546 1.105 wiz built in a subdirectory of KERNOBJDIR, which is typically
547 1.105 wiz sys/arch/MACHINE/compile or an associated object directory.
548 1.70 dholland
549 1.105 wiz This command does not imply the tools command; run the
550 1.105 wiz tools command first unless it is certain that the tools
551 1.70 dholland already exist and are up to date.
552 1.70 dholland
553 1.126 wiz This command will run "make cleandir" on the kernel in
554 1.105 wiz question first unless the -u option is given.
555 1.22 lukem
556 1.106 wiz kernel.gdb=kconf
557 1.106 wiz Build a new kernel with debug information. Similar to the
558 1.106 wiz above kernel=kconf operation, but creates a netbsd.gdb file
559 1.106 wiz alongside of the kernel netbsd, which contains a full
560 1.106 wiz symbol table and can be used for debugging (for example
561 1.106 wiz with a cross-gdb built by MKCROSSGDB).
562 1.106 wiz
563 1.113 uebayasi kernels This command will build all kernels defined in port
564 1.113 uebayasi specific release build procedure.
565 1.111 uebayasi
566 1.112 uebayasi This command internally calls the kernel=kconf operation
567 1.112 uebayasi for each found kernel configuration file.
568 1.111 uebayasi
569 1.105 wiz modules This command will build kernel modules and install them
570 1.105 wiz into DESTDIR.
571 1.86 jnemeth
572 1.105 wiz releasekernel=kconf
573 1.70 dholland Install a gzip(1)ed copy of the kernel previously built by
574 1.105 wiz kernel=kconf into
575 1.105 wiz RELEASEDIR/RELEASEMACHINEDIR/binary/kernel, usually as
576 1.126 wiz netbsd-kconf.gz, although the "netbsd" prefix is determined
577 1.126 wiz from the "config" directives in kconf.
578 1.24 lukem
579 1.126 wiz sets Perform "make sets".
580 1.31 lukem
581 1.126 wiz sourcesets Perform "make sourcesets".
582 1.22 lukem
583 1.126 wiz syspkgs Perform "make syspkgs".
584 1.54 apb
585 1.126 wiz iso-image Perform "make iso-image".
586 1.56 apb
587 1.105 wiz iso-image-source
588 1.126 wiz Perform "make iso-image-source".
589 1.65 jnemeth
590 1.105 wiz install-image
591 1.126 wiz Perform "make install-image".
592 1.97 tsutsui
593 1.126 wiz live-image Perform "make live-image".
594 1.97 tsutsui
595 1.148 lukem list-arch Show a list of valid MACHINE and MACHINE_ARCH settings, the
596 1.148 lukem default MACHINE_ARCH for each MACHINE, and aliases for
597 1.109 apb MACHINE/MACHINE_ARCH pairs, and then exits. The -m or -a
598 1.109 apb options (or both) may be used to specify glob patterns that
599 1.109 apb will be used to narrow the list of results; for example,
600 1.131 leot "build.sh -m 'evb*' -a '*arm*' list-arch" will list all
601 1.109 apb known MACHINE/MACHINE_ARCH values in which either MACHINE
602 1.109 apb or ALIAS matches the pattern `evb*', and MACHINE_ARCH
603 1.109 apb matches the pattern `*arm*'.
604 1.109 apb
605 1.105 wiz The following command line options alter the behaviour of the build.sh
606 1.51 wiz operations described above:
607 1.1 tv
608 1.109 apb -a arch Set the value of MACHINE_ARCH to arch. See the -m option for
609 1.109 apb more information.
610 1.9 thorpej
611 1.105 wiz -B buildid
612 1.105 wiz Set the value of BUILDID to buildid. This will also append the
613 1.159 lukem build identifier to the name of the nbmake-MACHINE wrapper
614 1.159 lukem script so that the resulting name is of the form
615 1.126 wiz "nbmake-MACHINE-BUILDID".
616 1.1 tv
617 1.105 wiz -C cdextras
618 1.105 wiz Append cdextras to the CDEXTRA variable, which is a space-
619 1.104 wiz separated list of files or directories that will be added to
620 1.126 wiz the CD-ROM image that may be create by the "iso-image" or
621 1.126 wiz "iso-image-source" operations. Files will be added to the root
622 1.126 wiz of the CD-ROM image, whereas directories will be copied
623 1.87 apb recursively. If relative paths are specified, they will be
624 1.87 apb converted to absolute paths before being used. Multiple paths
625 1.105 wiz may be specified via multiple -C options, or via a single
626 1.87 apb option whose argument contains multiple space-separated paths.
627 1.66 jnemeth
628 1.136 uwe -c compiler
629 1.136 uwe Select the compiler for the toolchain to build NetBSD and for
630 1.136 uwe inclusion in the NetBSD distribution. Supported choices:
631 1.136 uwe
632 1.136 uwe clang
633 1.136 uwe
634 1.136 uwe gcc [default]
635 1.136 uwe
636 1.136 uwe The compiler used to build the toolchain can be different; see
637 1.136 uwe HOST_CC and HOST_CXX.
638 1.136 uwe
639 1.105 wiz -D dest Set the value of DESTDIR to dest. If a relative path is
640 1.104 wiz specified, it will be converted to an absolute path before
641 1.104 wiz being used.
642 1.104 wiz
643 1.105 wiz -E Set `expert' mode. This overrides various sanity checks, and
644 1.105 wiz allows: DESTDIR does not have to be set to a non-root path for
645 1.105 wiz builds, and MKUNPRIVED=yes does not have to be set when
646 1.104 wiz building as a non-root user.
647 1.23 lukem
648 1.105 wiz Note: It is highly recommended that you know what you are doing
649 1.23 lukem when you use this option.
650 1.19 lukem
651 1.148 lukem -h Show a help message, and exit.
652 1.51 wiz
653 1.105 wiz -j njob Run up to njob make(1) subjobs in parallel; passed through to
654 1.74 apb make(1). If you see failures for reasons other than running
655 1.105 wiz out of memory while using build.sh with -j, please save
656 1.104 wiz complete build logs so the failures can be analyzed.
657 1.74 apb
658 1.105 wiz To achieve the fastest builds, -j values between (1 + the
659 1.104 wiz number of CPUs) and (2 * the number of CPUs) are recommended.
660 1.104 wiz Use lower values on machines with limited memory or I/O
661 1.104 wiz bandwidth.
662 1.104 wiz
663 1.126 wiz -M obj Set MAKEOBJDIRPREFIX to obj. Unsets MAKEOBJDIR. See "-O obj"
664 1.126 wiz for more information.
665 1.104 wiz
666 1.105 wiz For instance, if the source directory is /usr/src, a setting of
667 1.126 wiz "-M /usr/obj" will place build-time files under
668 1.105 wiz /usr/obj/usr/src/bin, /usr/obj/usr/src/lib,
669 1.105 wiz /usr/obj/usr/src/usr.bin, and so forth.
670 1.88 apb
671 1.73 apb If a relative path is specified, it will be converted to an
672 1.105 wiz absolute path before being used. build.sh imposes the
673 1.105 wiz restriction that the argument to the -M option must not begin
674 1.126 wiz with a "$" (dollar sign) character; otherwise it would be too
675 1.104 wiz difficult to determine whether the value is an absolute or a
676 1.104 wiz relative path. If the directory does not already exist,
677 1.105 wiz build.sh will create it.
678 1.18 lukem
679 1.109 apb -m mach Set the value of MACHINE to mach, unless the mach argument is
680 1.109 apb an alias that refers to a MACHINE/MACHINE_ARCH pair, in which
681 1.109 apb case both MACHINE and MACHINE_ARCH are set from the alias.
682 1.109 apb Such aliases are interpreted entirely by build.sh; they are not
683 1.109 apb used by any other part of the build system. The MACHINE_ARCH
684 1.109 apb setting implied by mach will override any value of MACHINE_ARCH
685 1.109 apb in the process environment, but will not override a value set
686 1.109 apb by the -a option. All cross builds require -m, but if unset on
687 1.109 apb a NetBSD host, the host's value of MACHINE will be detected and
688 1.109 apb used automatically.
689 1.109 apb
690 1.109 apb See the list-arch operation for a way to get a list of valid
691 1.109 apb MACHINE and MACHINE_ARCH settings.
692 1.1 tv
693 1.105 wiz -N noiselevel
694 1.126 wiz Set the "noisyness" level of the build, by setting MAKEVERBOSE
695 1.126 wiz to noiselevel.
696 1.45 lukem
697 1.105 wiz -n Show the commands that would be executed by build.sh, but do
698 1.126 wiz not make any changes. This is similar in concept to "make -n".
699 1.1 tv
700 1.105 wiz -O obj Create an appropriate transform macro for MAKEOBJDIR that will
701 1.105 wiz place the built object files under obj. Unsets
702 1.105 wiz MAKEOBJDIRPREFIX.
703 1.88 apb
704 1.126 wiz For instance, a setting of "-O /usr/obj" will place build-time
705 1.126 wiz files under /usr/obj/bin, /usr/obj/lib, /usr/obj/usr.bin, and
706 1.126 wiz so forth.
707 1.88 apb
708 1.88 apb If a relative path is specified, it will be converted to an
709 1.105 wiz absolute path before being used. build.sh imposes the
710 1.105 wiz restriction that the argument to the -O option must not contain
711 1.126 wiz a "$" (dollar sign) character. If the directory does not
712 1.105 wiz already exist, build.sh will create it.
713 1.73 apb
714 1.105 wiz In normal use, exactly one of the -M or -O options should be
715 1.105 wiz specified. If neither -M nor -O is specified, then a default
716 1.78 apb object directory will be chosen according to rules in
717 1.78 apb <bsd.obj.mk>. Relying on this default is not recommended
718 1.78 apb because it is determined by complex rules that are influenced
719 1.78 apb by the values of several variables and by the location of the
720 1.78 apb source directory.
721 1.18 lukem
722 1.151 lukem Note: Placing the obj directory location outside of the default
723 1.151 lukem source tree hierarchy makes it easier to manually clear out old
724 1.151 lukem files in the event the "make cleandir" operation is unable to
725 1.151 lukem do so. (See CAVEATS below.)
726 1.151 lukem
727 1.151 lukem Note: The use of one of -M or -O is the only means of building
728 1.151 lukem multiple machine architecture userlands from the same source
729 1.151 lukem tree without cleaning between builds (in which case, one would
730 1.151 lukem specify distinct obj locations for each).
731 1.109 apb
732 1.126 wiz -o Set the value of MKOBJDIRS to "no". Otherwise, it will be
733 1.126 wiz automatically set to "yes". This default is opposite to the
734 1.105 wiz behaviour when not using build.sh.
735 1.1 tv
736 1.148 lukem -P Set the value of MKREPRO and MKREPRO_TIMESTAMP to the latest
737 1.148 lukem source CVS timestamp for reproducible builds.
738 1.148 lukem
739 1.105 wiz -R rel Set the value of RELEASEDIR to rel. If a relative path is
740 1.59 apb specified, it will be converted to an absolute path before
741 1.59 apb being used.
742 1.18 lukem
743 1.105 wiz -r Remove the contents of DESTDIR and TOOLDIR before building
744 1.1 tv (provides a clean starting point). This will skip deleting
745 1.105 wiz DESTDIR if building on a native system to the root directory.
746 1.1 tv
747 1.105 wiz -S seed Change the value of BUILDSEED to seed. This should rarely be
748 1.72 perry necessary.
749 1.72 perry
750 1.105 wiz -T tools Set the value of TOOLDIR to tools. If a relative path is
751 1.104 wiz specified, it will be converted to an absolute path before
752 1.126 wiz being used. If set, the bootstrap "make" will only be rebuilt
753 1.126 wiz if the source files for make(1) have changed.
754 1.104 wiz
755 1.105 wiz -U Set MKUNPRIVED=yes.
756 1.104 wiz
757 1.105 wiz -u Set MKUPDATE=yes.
758 1.104 wiz
759 1.105 wiz -V var=[value]
760 1.105 wiz Set the environment variable var to an optional value. This is
761 1.159 lukem propagated to the nbmake-MACHINE wrapper script.
762 1.105 wiz
763 1.105 wiz -w wrapper
764 1.159 lukem Create the nbmake-MACHINE wrapper script (see below) in a
765 1.159 lukem custom location, specified by wrapper. This allows, for
766 1.159 lukem instance, to place the wrapper script in PATH automatically.
767 1.151 lukem
768 1.151 lukem Note: wrapper is the full name of the file, not just a
769 1.151 lukem directory name. If a relative path is specified, it will be
770 1.151 lukem converted to an absolute path before being used.
771 1.1 tv
772 1.105 wiz -X x11src
773 1.105 wiz Set the value of X11SRCDIR to x11src. If a relative path is
774 1.59 apb specified, it will be converted to an absolute path before
775 1.59 apb being used.
776 1.49 lukem
777 1.105 wiz -x Set MKX11=yes.
778 1.49 lukem
779 1.105 wiz -Z var Unset ("zap") the environment variable var. This is propagated
780 1.159 lukem to the nbmake-MACHINE wrapper script.
781 1.40 lukem
782 1.148 lukem -? Show a help message, and exit.
783 1.148 lukem
784 1.105 wiz The "nbmake-MACHINE" wrapper script
785 1.159 lukem If using the build.sh script to build NetBSD, a nbmake-MACHINE wrapper
786 1.159 lukem script will be created in TOOLDIR/bin upon the first build to assist in
787 1.159 lukem building subtrees on a cross-compile host.
788 1.159 lukem
789 1.159 lukem The nbmake-MACHINE wrapper script can be invoked in lieu of make(1), and
790 1.159 lukem will instead call the up-to-date version of "nbmake" installed into
791 1.159 lukem TOOLDIR/bin with several key variables pre-set, including MACHINE,
792 1.159 lukem MACHINE_ARCH, and TOOLDIR. nbmake-MACHINE will also set variables
793 1.159 lukem specified with -V, and unset variables specified with -Z. Note that by
794 1.159 lukem default these variables will not override mk.conf(5); see "make"
795 1.159 lukem variables for more details.
796 1.40 lukem
797 1.159 lukem This wrapper script can be symlinked into a directory listed in PATH, or
798 1.159 lukem called with an absolute path.
799 1.1 tv
800 1.105 wiz EXAMPLES
801 1.151 lukem 1. % ./build.sh [OPTIONS] tools kernel=GENERIC
802 1.28 lukem
803 1.28 lukem Build a new toolchain, and use the new toolchain to configure and
804 1.28 lukem build a new GENERIC kernel.
805 1.28 lukem
806 1.151 lukem 2. % ./build.sh [OPTIONS] -U distribution
807 1.28 lukem
808 1.105 wiz Using unprivileged mode, build a complete distribution to a DESTDIR
809 1.148 lukem directory that build.sh selects (and will show).
810 1.28 lukem
811 1.151 lukem 3. # ./build.sh [OPTIONS] -U install=/
812 1.28 lukem
813 1.105 wiz As root, install to / the distribution that was built by example 2.
814 1.105 wiz Even though this is run as root, -U is required so that the
815 1.105 wiz permissions stored in DESTDIR/METALOG are correctly applied to the
816 1.105 wiz files as they're copied to /.
817 1.28 lukem
818 1.151 lukem 4. % ./build.sh [OPTIONS] -U -u release
819 1.28 lukem
820 1.105 wiz Using unprivileged mode, build a complete release to DESTDIR and
821 1.148 lukem RELEASEDIR directories that build.sh selects (and will show).
822 1.126 wiz MKUPDATE=yes (-u) is set to prevent the "make cleandir", so that if
823 1.126 wiz this is run after example 2, it doesn't need to redo that portion of
824 1.126 wiz the release build.
825 1.16 lukem
826 1.105 wiz SEE ALSO
827 1.155 lukem config(1), ctags(1), ex(1), gzip(1), lint(1), make(1), mandoc(1),
828 1.155 lukem mkisofs(1), nroff(1), vi(1), mk.conf(5), atf(7), hier(7), mdoc(7),
829 1.155 lukem release(7), etcupdate(8), installboot(8), mount(8), postinstall(8),
830 1.151 lukem sysinst(8), pkgsrc/sysutils/cdrtools
831 1.1 tv
832 1.155 lukem Note: The NetBSD manual pages are also available at
833 1.155 lukem https://man.netbsd.org
834 1.155 lukem
835 1.105 wiz HISTORY
836 1.105 wiz The build.sh based build scheme was introduced for NetBSD 1.6 as
837 1.105 wiz USE_NEW_TOOLCHAIN, and re-worked to TOOLCHAIN_MISSING after that.
838 1.1 tv
839 1.109 apb CAVEATS
840 1.109 apb After significant updates to third-party components in the source tree,
841 1.126 wiz the "make cleandir" operation may be insufficient to clean out old files
842 1.126 wiz in object directories. Instead, one may have to manually remove the
843 1.126 wiz files. Consult the UPDATING file for notices concerning this.
844 1.109 apb
845 1.159 lukem NetBSD July 21, 2023 NetBSD
846