Home | History | Annotate | Download | only in make
History log of /src/usr.bin/make/make.h
RevisionDateAuthorComments
 1.361  06-Jul-2025  rillig make: clean up
 1.360  13-Jun-2025  rillig make: skip inter-process stack trace when MAKE_STACK_TRACE=no

Previously, setting MAKE_STACK_TRACE to any other value than "yes"
accidentally generated almost-complete inter-process stack traces.
 1.359  13-Jun-2025  rillig make: add on-demand inter-process stack traces

When debugging a build that heavily uses nested calls to sub-makes, the
chain of calls is useful to quickly assess the situation. If explicitly
requested, include the actions from any parent processes in the stack
traces.

Reviewed by: sjg
 1.358  11-Jun-2025  sjg make: check variable names for invalid characters

Variable names should not contain white-space.
As is; make quietly ignores gmake syntax like
$(addprefix -I, $(INCLUDE_DIR_LIST))
which means we are not doing what the makefile wants, and the
user has no clue.

CheckVarname is called only if there are no modifiers to be applied
as :L and others legitimately can handle white-space,
it will issue a warning if illegal characters are found.
Currently only white-space is checked.

The NetBSD build produces a few warnings - in one case a makefile
intended for gmake, and the other some dubious constructs by bsd.prog.mk
in support of ATF tests such as OBJS.${PROG} when PROG is actually a list.

Reviewed by: rillig
 1.357  26-May-2025  rillig make: prepare for inter-process stack traces

To understand how chains of sub-makes interact and how they are called,
it's useful to have stack traces that span multiple processes. To
implement this, a stack trace needs not only be printed but also stored
as a string.
 1.356  26-May-2025  rillig make: show contents of MAKEFLAGS in the stack trace

When the internal -J option refers to a closed file descriptor, it's
helpful to know what the MAKEFLAGS environment variable contains, in
addition to the executed command line. Same for parse errors in
variable assignments from the command line.
 1.355  18-May-2025  rillig make: rename variables, remove now-redundant comments
 1.354  09-May-2025  rillig make: replace bitset in trace output with descriptive node attributes
 1.353  22-Apr-2025  rillig make: clean up

Replace 'unsigned int' with simply 'unsigned'.

In compat.c, skipping whitespace is not needed, as the loop above
already skips it.

In job.c, remove the unused header <sys/file.h>.

Inline the TMPPAT macro, as it is only needed in a single place.
 1.352  30-Mar-2025  sjg make: POSIX mode, check for sysV style modifiers first

A POSIX compatible makefile should not be using any
of our native modifiers, so give preference to the sysV style modifier
and only check the others as a fallback.

Reviewed by: rillig
 1.351  29-Mar-2025  rillig make: in a stack trace with details and includes, list the current file
 1.350  07-Mar-2025  rillig make: clean up comments and code for parallel mode

Most of the comments didn't help understand the code.

The Finish function was named way too soft for what it does, which is to
exit without cleaning up properly. Since it was used only once, inline
it. The code from that function causes jobs to be terminated early when
an unrelated target has failed, resulting in partially written files,
due to SIGPIPE.
 1.349  19-Jan-2025  rillig make: fix code coverage counts

See tests/usr.bin/gcov/t_gcov.sh.
 1.348  11-Jan-2025  rillig make: replace "Malformed conditional" with "Variable is undefined"

Evaluating a conditional such as ".if ${UNDEF}" produced a "Malformed
conditional" error, even though the form of the conditional was correct,
it was the content of the variable that was wrong.

Replace this message with the more accurate "Variable is undefined",
revealing which of the possibly many variables is actually undefined.
 1.347  29-Aug-2024  rillig make: reduce line length in error messages

The error messages for deeply nested parse errors were hard to decipher,
due to the large amount of text. Split these messages into individual
lines, just as in the backtrace for .include files and .for loops. This
unified backtrace makes the output more uniform.
 1.346  27-Aug-2024  rillig make: treat recursive variables non-fatally

A recursive variable is no worse than an unknown modifier, so treat them
in the same way by continuing parsing until the end of the makefile.
 1.345  25-Aug-2024  rillig make: add more context to error message about recursive variables
 1.344  11-Jul-2024  sjg Compat_RunCommand use tempfile if cmd too big

Extract the logic recently added to Cmd_Exec to handle
long commands via temp file to Cmd_Argv,
so it can also be leveraged by Compat_RunCommand

Reviewed by: christos
 1.343  09-Jul-2024  rillig make: error out on parse/evaluation errors in shell commands

The expression ${VAR:X} has an unknown modifier ':X'. Previously, this
expression errored out when the expression was evaluated at parse time,
but not when the expression was evaluated when generating the commands
to bring a target up to date. The errors were previously reported, they
didn't affect the exit status, though.

Now, errors in expressions are handled in the same way, regardless of
the time at which they are evaluated.
 1.342  07-Jul-2024  rillig make: move initialization of variable scopes to targ.c

The variable scopes are freed by Targ_End, so initialize them there as
well. Separate printing statistics and freeing memory, which makes
Var_End unnecessary.
 1.341  07-Jul-2024  rillig make: only generate code for cleanup functions in CLEANUP mode
 1.340  05-Jul-2024  rillig make: reduce lint-specific comments about ARGSUSED
 1.339  15-Jun-2024  rillig make: clean up collection of context information for error messages
 1.338  02-Jun-2024  rillig make: sync VarEvalMode constant names with their debug log names
 1.337  01-Jun-2024  sjg make: add .export-all

An explicit syntax for exporting all global variables is much safer
than allowing .export with no argument to do the same.

Add .export-all and have .export with no argument throw a warning saying
to use .export-all

Reviewed by: rillig
 1.336  01-Jun-2024  rillig make: remove unused VARE_EVAL_KEEP_DOLLAR
 1.335  25-May-2024  rillig make: fix memory leak for command strings
 1.334  25-May-2024  rillig make: fix a few more memory leaks
 1.333  07-May-2024  sjg make: all command line overrides go in .MAKEOVERRIDES

Not all variables that start with '.' are internals,
and unless they are explicitly flagged as internal should go
into .MAKEOVERRIDES

Update varname-dot-makeoverrides to check this.

Also avoid using SCOPE_CMDLINE when ReadOnly will do.
 1.332  27-Apr-2024  rillig make: clean up, test .NOPATH

Trim down the comments in the archive module, as they mainly repeated
the code. Trim down the binary code size in the archive module, as it
is rarely used.

In Var_Parse, delay two variable assignments until they are actually
needed.
 1.331  23-Apr-2024  rillig make: clean up comments, code and tests
 1.330  20-Apr-2024  rillig make: provide more context information for parse/evaluate errors
 1.329  10-Mar-2024  sjg make: record exit status in GNode

SetErrorVars can now set .ERROR_EXIT which allows
a .ERROR target to ignore the case of .ERROR_EXIT == 6
which means failure happened elsewhere.

Reviewed by:
 1.328  01-Mar-2024  sjg make: export target scope values

Pass target scope to Var_ReexportVars so that a target process
will see the correct values in its env.
We must then mark any Global scope variable as unexported
so targets without local value get the Global one.
 1.327  17-Dec-2023  rillig make: clean up unused return value of str2Lst_Append

No functional change.
 1.326  02-Nov-2023  rillig make: inline a single-line cross-file function

No functional change.
 1.325  10-Sep-2023  rillig make: remove redundant 'extern' from function declaration
 1.324  24-Jun-2023  rillig make: remove redundant 'extern' in function declaration

No binary change.
 1.323  20-Jun-2023  rillig make: allow targets to be used as multiple-inclusion guards

This style is used by FreeBSD, among others.
 1.322  19-Jun-2023  rillig make: if a makefile is protected by a guard, only include it once

"looks reasonable" sjg@
 1.321  16-Jun-2023  rillig make: remove parameter names from function declarations

No binary change.
 1.320  01-Jun-2023  rillig make: shorten function names, clean up comments

No functional change.
 1.319  28-Mar-2023  rillig make: declare all common symbols in headers, unexport others

No functional change.
 1.318  09-Mar-2023  rillig make: document how read-only variables can be made read-write
 1.317  18-Feb-2023  rillig make: fix parsing of unevaluated subexpressions with unbalanced '{}'

Since var.c 1.323 from 2020-07-26, modifiers containing unbalanced
braces or parentheses were parsed differently, depending on whether they
were relevant or not.

For example, the expression '${VAR:...}' is enclosed with braces. When
this expression has a modifier ':S,},}},g' that would double each '}' in
that expression, the parser got confused:

If the expression was relevant, the modifier was parsed as usual, taking
into account that the 3 '}' in the modifier are ordinary characters.

If the expression was irrelevant, the parser only counted the '{' and
the '}', without taking into account that a '}' might be escaped by a
'\' or be an ordinary character. Parsing therefore stopped at the first
'}', assuming it would finish the expression '${VAR:S,}'.

This parsing mode of only counting balanced '{' and '}' makes sense for
the modifier ':@var@...@', which expands each word of the expression
using the template from the '...'. These templates tend to be simple
enough that counting the '{' and '}' suffices.
 1.316  15-Feb-2023  rillig make: inline macros for variable names

The variable name '.MAKEOVERRIDES' was already used in the non-macro
form.

No binary change.
 1.315  15-Feb-2023  rillig make: inline macro for variable name ".MAKE.EXPORTED"

The variable name is distinctive enough to be searched directly in the
code instead of having a named constant for it.

No binary change.
 1.314  14-Feb-2023  rillig make: remove redundant type VarParseResult

No functional change.
 1.313  14-Feb-2023  rillig make: clean up calls to Var_Subst

None of the calls to Var_Subst used the return value, and the return
value was always VPR_OK.

No functional change.
 1.312  14-Feb-2023  rillig make: reduce complexity of evaluating expressions

No functional change.
 1.311  26-Jan-2023  sjg make: some variables should be read-only

Make variables like .newline and .MAKE.{GID,PID,PPID,UID} read-only.

Reviewed by: rillig
 1.310  23-Jan-2023  sjg make: .[NO]READONLY for control of read-only variables

Reviewed by: rillig
 1.309  19-Jan-2023  rillig make: inline macro for variable name

This fixes the inconsistency of using the macro name in one place and
its value in another place (since 2010).

No binary change.
 1.308  10-Oct-2022  rillig make: change return type of unlink_file back to int

As unlink_file is a wrapper around unlink, use the same encoding for the
possible return values as in the wrapped function. This consistency is
more important than expressing all possible return values in the return
type 'bool'.

https://mail-index.netbsd.org/tech-toolchain/2022/10/06/msg004155.html

No functional change.
 1.307  24-Sep-2022  rillig make: clean up tracking of depth of nested .if directives

The variable cond_min_depth was redundant. It was only accessed while
parsing the makefiles. Merging it into struct IncludedFile removes the
possible confusion between cond_min_depth and including_cond_min_depth.

No functional change.
 1.306  24-Sep-2022  rillig make: fix variable and function names relating to .if nesting

The previous names were confusing since they suggested that cond_depth
instead of cond_min_depth would be saved and restored.

No functional change.
 1.305  23-Sep-2022  sjg Cond_reset_depth just use cond_min_depth

To avoid errors from unclosed conditionals on .break
it is sufficient to just set cond_depth = cond_min_depth.

Patch from rillig
 1.304  02-Sep-2022  sjg make: add .break to terminate .for loop early

When .break is encountered within a .for loop
it causes immediate termination.

Outside of a .for loop .break causes a parse error.

Reviewed by: christos
 1.303  12-Jun-2022  rillig make: reorganize Parse_Error

Determining the location where the error occurred is now done by
ParseVErrorInternal. This frees the remaining code from keeping the
filename and the line number together. It also makes Parse_Error short
enough that it might be worth providing a separate function for each of
the 3 log levels.

No functional change.
 1.302  07-May-2022  rillig make: allow to randomize build order of targets

In complex dependency structures, when a build fails, a probable cause
is a missing dependency declaration between some files. In compat mode,
the build order is deterministic, in jobs mode, it is somewhat
deterministic. To explore more edge cases, add the line ".MAKE.MODE +=
randomize-targets" somewhere in the makefile.

Fixes PR bin/45226 by riastradh. Reviewed by christos.
 1.301  07-May-2022  rillig make: rename Compat_Run to Compat_MakeAll

No functional change.
 1.300  18-Apr-2022  rillig make: only switch to POSIX mode if '.POSIX:' is the first line

https://pubs.opengroup.org/onlinepubs/9699919799/utilities/make.html
says that in order to make a makefile POSIX-conforming, its first
non-comment line must be the special dependency line '.POSIX:' without
any source dependencies.

Previously, make switched to POSIX mode even if such a line occurred
anywhere else, which was allowed by POSIX but was deep in the
"unspecified behavior" area. For NetBSD make, there is no big
difference since it doesn't ship any <posix.mk> file, this change mainly
affects the bmake distribution.

Previously, makefiles that contain '.POSIX:' somewhere in the middle
could fail due to <posix.mk> resetting .SUFFIXES, among other things.

Suggested by Simon J. Gerraty, who also reviewed an earlier version of
this change.
 1.299  26-Mar-2022  rillig make: prefer 'long long' over 'long' on 32-bit C99 platforms

When sorting the words of an expression numerically using the modifier
':On' (added on 2021-07-30), use 64-bit numbers even on 32-bit
platforms. A typical use case is comparing file sizes.

When tracing the execution of jobs, fix an integer overflow after 2038.
32-bit platforms that use a pre-C99 compiler still have this problem.

No change to the test suite since most tests simply skip any potential
differences between 32-bit platforms and 64-bit platforms (see
varmod-order-numeric.mk) or already account for both variants (see
varmod-localtime.mk).
 1.298  05-Feb-2022  rillig make: improve C90 support

Do not use inline functions, remove trailing comma in enum declaration,
do not use 'long long' for printing a timestamp. This re-introduces the
Year 2038 Problem for pre-C99 compilers when printing the trace log, but
that is a seldom used feature.
 1.297  04-Feb-2022  rillig make: use unsigned int for line numbers everywhere

Previously, some line numbers were stored as signed int while others
were stored as size_t. Since line numbers are never negative, use an
unsigned type. Since the maximum file size for makefiles is 1 GB (see
loadfile), unsigned int is large enough even on 64-bit platforms.

Using a single data types reduces the number of type conversions. Using
unsigned int improves compatibility with C90 (printf %u instead of %zu),
which is needed by bmake, which is derived from usr.bin/make.

No functional change.
 1.296  31-Jan-2022  rillig make: make DEBUG0 simpler

All arguments to DEBUG0 are string literals, and none of them contains a
'%', which makes them safe to pass to printf directly. Any accidental
'%' would be caught by the compiler.

No functional change.
 1.295  30-Jan-2022  christos Make the GNode lineno unsigned to fix lint warning in var.c calling
PrintLocation()
 1.294  29-Jan-2022  rillig make: for recursive variables in commands, print location

Print the approximate location based on the last command that has been
defined for the target. It would be possible to get more detailed
location information by counting the number of commands of the target,
but that would get messy due to .USEBEFORE, .USE and .DEFAULT, and
still, this is an edge case, so don't waste too much code for it now.
Having this hint about the location is more helpful than just a plain
"Variable X is recursive" without any further details.
 1.293  29-Jan-2022  rillig make: print stack trace on fatal errors

The only fatal error that occurs while the makefiles are read in is the
one about recursive variables, which didn't give any hint about the
location before.

If a recursive variable is detected while evaluating the commands of a
target to be made, there is no location information, as before.
 1.292  29-Jan-2022  rillig make: inline functions that have a single call site

No functional change.
 1.291  27-Jan-2022  sjg Allow local variable assignments in dependency lines

The variable is set in the context of the target.
This syntax has been supported by gmake for ~ever.
If necessary a makefile can set .MAKE.TARGET_LOCAL_VARIABLES=false
to disable this.

Expose GetBooleanExpr so parse.c can use it.
 1.290  23-Jan-2022  rillig make: merge nonints.h into make.h

No functional change.
 1.289  15-Jan-2022  rillig make: use islower for parsing directives

None of the directives has an uppercase letter, so there is no need to
test for it.

No functional change.
 1.288  09-Jan-2022  rillig make: extract low-level character operations into utility function

Suggested by nia.
https://mail-index.netbsd.org/source-changes-d/2022/01/09/msg013564.html

No functional change.
 1.287  07-Jan-2022  rillig make: merge duplicate variables for the main target

No functional change.
 1.286  07-Jan-2022  rillig make: rename and inline Targ_Precious

No functional change.
 1.285  07-Jan-2022  rillig make: rename IFile and its fields to match their actual content

For lines that use backslash continuation, the human-readable line
number does not equal the number of raw lines that have been read from
the file.

The big comment in PrintStackTrace has become outdated, it still
referred to first_lineno. Due to the bugs documented in
opt-debug-parse.mk, that function needs to be redone completely.

No functional change.
 1.284  29-Dec-2021  rillig make: merge duplicate types CondEvalResult and CondResult

No binary change.
 1.283  29-Dec-2021  rillig make: merge duplicate enum constants for CondEvalResult and CondResult

No binary change.
 1.282  29-Dec-2021  rillig make: merge types CondResult and CondEvalResult

No functional change.
 1.281  28-Dec-2021  rillig make: do not treat .USEBEFORE as candidate for the main target

A .USE target is not a candidate, so .USEBEFORE shouldn't either.

Since make.h 1.36 from 2001-07-03. In that commit, OP_USEBEFORE should
have been added to OP_NOTARGET.
 1.280  28-Dec-2021  rillig make: extract OP_NOTARGET into separate function

No binary change, except for line numbers in assertions.
 1.279  27-Dec-2021  rillig make: remove unnecessary words from command line options

Several years ago, the command line options were individual global
variables. The global variable could therefore not be named 'silent'
since that would have conflicted with local variables of the same name.
After moving the global variable to the namespace 'struct CmdOpts',
there is no conflict anymore.

There doesn't seem to be any risk of naming collisions for the names
'touch' and 'query'.

No functional change.
 1.278  15-Dec-2021  rillig make: amend leftover cleanups from the previous commits

No functional change.
 1.277  15-Dec-2021  rillig make: format comments according to /usr/share/misc/style

Assisted by indent(1), with manual corrections due to its many remaining
bugs.

No functional change.
 1.276  15-Dec-2021  rillig make: remove redundant comments for multiple-inclusion guards
 1.275  15-Dec-2021  rillig make: remove space after ':' in bit-field declarations

As seen in /usr/share/misc/style.
 1.274  15-Dec-2021  rillig make: mark several functions whose result must be used

Suggested by sjg, to catch more bugs like the memory leak in cond.c
1.303 from 2021-12-13.

No binary change.
 1.273  15-Dec-2021  rillig make: prevent memory leaks from buffers

The warning about unused function results would have prevented the
memory leak that was fixed in cond.c 1.303 from 2021-12-13.
 1.272  13-Dec-2021  rillig make: convert debugging flags from enum to bit-field

This gets rid of the magic numbers, making it possible to add another
debug flag without renumbering the others.

No functional change.
 1.271  13-Dec-2021  rillig make: fix memory leak for filenames in .for loops (since 2013-06-18)

Previously, each time a .for directive pushed its buffer on the input
file stack, the current filename was duplicated. This was a waste of
memory.

The name of a file is typically only used while it is read in. There is
one situation when the filename is needed for longer, which is when a
target is defined.

Since .for loops are implemented as a special form of included files,
each .for loop duplicated the current filename as well.

$ cat << EOF > for.mk
.for i in 1 2 3 4 5 6 7 8 9 0
.for i in 1 2 3 4 5 6 7 8 9 0
.for i in 1 2 3 4 5 6 7 8 9 0
.for i in 1 2 3 4 5 6 7 8 9 0
.for i in 1 2 3 4 5 6 7 8 9 0
.for i in 1 2 3 4 5 6 7 8 9 0
.for i in 1 2 3 4 5 6 7 8 9 0
.endfor
.endfor
.endfor
.endfor
.endfor
.endfor
.endfor

all:
@ps -o rsz -p ${.MAKE.PID}
EOF

$ make-2021.12.13.03.55.16 -r -f for.mk
RSZ
10720

$ ./make -r -f for.mk
RSZ
1716

The difference is 8 MB, which amounts to 1 million .for loops.
 1.270  28-Nov-2021  rillig make: fix a few lint warnings about type mismatch in enum comparisons

These warnings were triggered with the lint flag '-e', which enables
additional checks on enums. This check would have detected the type
mismatch from the previous commit.

The check has a few strange warnings though, complaining about
initialization of 'unsigned long' with 'unsigned long', so don't enable
it for the official builds.

No functional change.
 1.269  28-Nov-2021  rillig make: fix leftover typo from previous refactoring
 1.268  28-Nov-2021  rillig make: convert GNodeFlags from enum into bit-fields

Now that Enum_ToString is implemented for each type separately, it's
easy to convert them to bit-fields. This gets rid of the magic numbers
12 for CYCLE and 13 for DONECYCLE that left a suspicious gap in the
numbers. This gap was not needed since the code didn't make use of the
relative ordering of the enum constants.

The effects of this conversion are fewer capital letters in the code,
smaller scope for the GNode flags, and clearer code especially when
setting a flag back to false.

One strange thing is that GCC 10.3.0 doesn't optimize GNodeFlags_IsNone
to an single bitmasking instruction, at least on x86_64. Instead it
generates a testb instruction for each of the flags, even loading bit 8
separately from the others. Clang 12.0.1 knows this optimization
though and generates the obvious sequence of movzwl, testl, jz.

No functional change.
 1.267  28-Nov-2021  rillig make: replace bloated bit-set-to-string code with simple code

It was a nice idea to implement a bit-set using an enum type and have a
generic ToString function for them. In the end, the implementation
involved really heavy preprocessor magic and was probably difficult to
understand. Replace all the code with a few bits of straight-forward
preprocessor magic that can be readily understood by just looking 5
lines around, instead of digging through 130 lines of lengthy macro
definitions.

Curiously, this reduces the binary size even though the 3 ToString
functions now have a few lines of duplicate code and there are more
explicit function calls.

The ToString functions are only seldom used, so the additional memory
allocation is acceptable.

No functional change.
 1.266  27-Nov-2021  rillig make: remove CONSTCOND comments, lint no longer needs them
 1.265  12-Sep-2021  rillig make: error out if a pre-C99 platform defines bool in some cases

On NetBSD/amd64 9.99.88, when compiling make in C90 mode, make.h defined
its own boolean type as an alias for unsigned int. Not plain int since
that would make the value of bit-fields -1 instead of 1.

This worked fine for all files except main.c, which includes
<sys/sysctl.h>, which in turn includes <stdbool.h> unconditionally, even
in C90 mode. This meant that in main.c, sizeof(bool) was 1, while in
all other files it was 4.

This in turn led to a segmentation fault when ParseDependencySourceMain
tried to access opts.create. Since parse.c assumed sizeof(bool) == 4,
it computed an offset outside of struct CmdOpts, which was defined in
main.c with sizeof(bool) == 1.

Rather than risking these segmentation faults, prevent building make on
platforms like these and suggest a proper workaround.
 1.264  31-Jul-2021  rillig make: fix lint warnings

The string functions from str.h are declared as 'static __unused' when
compiled with GCC, but lint explicitly undefines __GCC__ during
preprocessing. Therefore, make those functions inline, to prevent
warnings that they are unused.

The macro UNCONST is used in a few places, and (again) since lint
undefines __GCC__, that macro expanded to a simple type cast, which lint
warned about. To prevent this warning, implement UNCONST as a function
that works everywhere and hides the type cast.

In filemon_open, the code for closing F->in was obviously unreachable.

No functional change.
 1.263  21-Jun-2021  rillig make: document where to find tests for the dependency lines
 1.262  14-Apr-2021  rillig make: let the compiler decide whether to inline string functions

On x86_64, this reduces the binary size by 2 kB.
 1.261  11-Apr-2021  rillig make: add types Substring and LazyBuf

These will be used for making the string handling more efficient,
avoiding allocations, especially when evaluating variable expressions.

Since the string handling has grown quite a bit in the last months,
extract it into its own header file.

No functional change.
 1.260  04-Apr-2021  rillig make: rename a few functions to be more descriptive

No functional change.
 1.259  03-Apr-2021  rillig make: revert accidental change from the previous commit

The definition of MAKE_GNUC_PREREQ was not supposed to be changed. This
change only slipped accidentally, the test for __STDC_VERSION__ was only
supposed to be for UNCONST.
 1.258  03-Apr-2021  rillig make: backport to C90

In the past few months I had accidentally used C99 features in the make
code. According to tools/README, tools that are used in the build
system should restrict themselves to C90.

This allows make to build with GCC's options "-pedantic
-Wno-system-headers -Dinline= -Wno-error=cast-qual".

I didn't notice anyone actively complaining though, I just wanted to see
how much work this backporting would be. The identifier __func__ is
still used, as in other tools.

No functional change.
 1.257  03-Apr-2021  rillig make: use C99 bool type instead of defining its own

No functional change.
 1.256  05-Feb-2021  sjg Avoid strdup in mkTempFile

Require caller to pass a buffer and size if they
want the tempfile not unlinked.

Add Job_TempFile to handle blocking signals around
call to mkTempFile, so that meta_open_filemon can use it
in jobs mode.
 1.255  05-Feb-2021  rillig make: improve documentation about variable scopes

In an experiment, I tried to separate the concepts of a GNode and a
variable scope. The global variables SCOPE_GLOBAL, SCOPE_INTERNAL and
SCOPE_CMDLINE are implemented as GNode even though they only need the
members 'name' and 'vars'. All their other members are unused.
Therefore it seemed natural to extract this part of the GNode into a
separate type called Scope.

The resulting code was harder to read though since it had split the
namespace of the functions into several parts that were not obviously
related: The Var_ functions, the Scope_ functions, and the short-cut
Global_ functions. Because of this, I threw away the experiment.

All that is left are a few updated comments.
 1.254  04-Feb-2021  rillig make: rename Var_ValueDirect to GNode_ValueDirect
 1.253  04-Feb-2021  rillig make: rename context and ctxt to scope

This continues the previous commit, in which VAR_GLOBAL, VAR_INTERNAL
and VAR_CMDLINE were renamed.

Renaming the variable 'ctxt' was trivial since that word is used nowhere
else. In the comments though, each occurrence of the word 'context' had
to be checked individually since the word 'context' was not only used
for referring to a variable scope. It is also used to distinguish
different situations where characters are escaped in a certain way
('parsing context') and in a few other expressions.
 1.252  04-Feb-2021  rillig make: rename some VAR constants to SCOPE

The word "context" does not fit perfectly to the variables that are
associate with a GNode, as the context is usually something from the
outside and the variables are more like properties inherent to the
GNode.

The term "global context" fits even less. Since the thing where
variables are looked up is commonly named a scope, use that term
instead.

This commit only renames the global variables VAR_GLOBAL, VAR_INTERNAL
and VAR_CMDLINE, plus a few very closely related comments. These are:

GNode.vars (because of line breaks)
GNode_Free (dito)
varname-make_print_var_on_error.mk
varname-make_print_var_on_error-jobs.mk

The debug message in Var_Stats is left as-is since there is no unit test
for it yet.

The other renamings (variable names "context", "ctxt", as well as
further comments) will be done in a follow-up commit.
 1.251  02-Feb-2021  rillig make: remove unused INTERNAL flag

It had been used for cached_realpaths, until this variable had its type
changed from GNode to HashTable in main.c 1.469 from 2020-11-14.
 1.250  01-Feb-2021  rillig make: clean up comments
 1.249  01-Feb-2021  rillig make: indent preprocessor directives consistently

As seen in share/misc/style.
 1.248  01-Feb-2021  rillig make: always use vfork, never fork

Before compat.c 1.217, job.c 1.390 and main.c 1.504 from 2020-12-27, the
exported make variables were exported from each freshly forked child
process. There was no practical difference though between exporting the
variables from the parent process or the child process since these two
processes share the same address space, except that the forked process
is very limited in what it may actually do. This limitation was
violated on a regular basis.

When an exported variable referred to a variable that used the :sh
variable modifier, this led to a fork from within vfork, which is not
allowed. Since 2020-12-27, exporting the variables is done from the
main process, which prevents this situation from ever occurring.

Since that day, there is no need anymore to distinguish between vfork
and fork, which removes any need for the macro.
 1.247  01-Feb-2021  rillig make: use bit shifts in enum constants for GNodeFlags

Same as in the other enum bit sets. This makes it easier to spot the
gap between bit 6 and bit 12.
 1.246  24-Jan-2021  rillig make(1): convert SearchPath to struct

This prepares for making dotLast a simple struct member instead of a
fake CachedDir, which is easier to understand.
 1.245  21-Jan-2021  rillig make(1): merge duplicate code in Parse_MainName
 1.244  19-Jan-2021  rillig make(1): remove do-not-format markers from comments

These markers had been used inconsistently. Furthermore the source code
had not been formatted automatically before 2020 at all, otherwise there
wouldn't have been any trailing whitespace left.
 1.243  16-Jan-2021  rillig make(1): fix a few inconsistencies for lint's strict bool mode
 1.242  10-Jan-2021  rillig make(1): consistently use boolean expressions in conditions

Most of the make code already followed the style of explicitly writing
(ptr != NULL) instead of the shorter (ptr) in conditions.

The remaining 50 instances have been found by an experimental,
unpublished check in lint(1) that treats bool expressions as
incompatible to any other scalar type, just as in Java, C#, Pascal and
several other languages.

The only unsafe operation on Boolean that is left over is (flags &
FLAG), for an enum implementing a bit set. If Boolean is an ordinary
integer type (the default), some high bits may get lost. But if Boolean
is the same as _Bool (by compiling with -DUSE_C99_BOOLEAN), C99 6.3.1.2
defines that a conversion from any scalar to the type _Bool acts as a
comparison to 0, which cannot lose any bits.
 1.241  30-Dec-2020  rillig make(1): format multi-line comments
 1.240  28-Dec-2020  rillig make(1): replace global preserveUndefined with VARE_KEEP_UNDEF

Controlling the expansion of variable expressions using a global
variable and a VARE flag was inconsistent.

Converting the global variable into a flag had to prerequisites:

1. The unintended duplicate variable assignment had to be fixed, as
done in parse.c 1.520 from 2020-12-27. Without this fix, it would have
been necessary to add more flags to Var_Exists and Var_SetWithFlags, and
this would have become too complex.

2. There had to be a unit test demonstrating that VARE_KEEP_DOLLAR only
applies to the top-level expression and is not passed to the
subexpressions, while VARE_KEEP_UNDEF applies to all subexpressions as
well. This test is in var-op-expand.mk 1.10 from 2020-12-28, at least
for the ':@word@' modifier. In ParseModifierPartSubst, VARE_KEEP_UNDEF
is not passed down either, in the same way.
 1.239  23-Dec-2020  rillig make(1): fix lint warnings for constant condition in DEBUG calls
 1.238  23-Dec-2020  rillig make(1): fix MAKE_RCSID for lint mode

Previously, running lint mode didn't define MAKE_RCSID at all, which
resulted in a syntax error.

While here, reduced the indentation and nesting of the preprocessor
directives.
 1.237  23-Dec-2020  rillig make(1): rename CmdOpts.lint to strict

When running lint(1) on the code, it defines the preprocessor macro
"lint" to 1, which generated a syntax error in the declaration "Boolean
lint", as that became "Boolean 1".
 1.236  22-Dec-2020  rillig make(1): fix return type of macro DEBUG

This macro was supposed to return a boolean expression all the time, it
just hadn't been implemented this way. This resulted in wrong output
for the test sh-flags, in compilation modes -DUSE_UCHAR_BOOLEAN and
-DUSE_CHAR_BOOLEAN, since in ParseCommandFlags, the expression
DEBUG(LOUD) didn't fit into a boolean.
 1.235  18-Dec-2020  rillig make(1): support using C99 bool for Boolean
 1.234  13-Dec-2020  rillig make(1): constify progname
 1.233  11-Dec-2020  rillig make(1): clean up comments for command line options
 1.232  06-Dec-2020  rillig make(1): clean up macros for debug logging

Using a do-while loop prevents compiler warnings about possible dangling
else. It also removes the unnecessary negation.
 1.231  05-Dec-2020  rillig make(1): define constants for enum zero-values
 1.230  29-Nov-2020  rillig make(1): use space instead of tab for preprocessor directives
 1.229  29-Nov-2020  rillig make(1): reduce memory allocation for dirSearchPath
 1.228  28-Nov-2020  rillig make(1): reduce memory allocation for CmdOpts.create
 1.227  28-Nov-2020  rillig make(1): reduce memory allocation in CmdOpts.variables
 1.226  28-Nov-2020  rillig make(1): reduce memory allocation in CmdOpts.makefiles
 1.225  28-Nov-2020  rillig make(1): reduce pointer indirection for GNode.implicitParents
 1.224  28-Nov-2020  rillig make(1): reduce pointer indirection for GNode.cohorts
 1.223  28-Nov-2020  rillig make(1): reduce pointer indirection for GNode.order_pred and order_succ
 1.222  28-Nov-2020  rillig make(1): reduce memory allocation for GNode.parents and GNode.children
 1.221  28-Nov-2020  rillig make(1): remove pointer indirection from GNode.commands

Just to save a few memory allocations. No noticeable effect on the
performance though.
 1.220  28-Nov-2020  rillig make(1): allow multiple RCS IDs in all-in-one mode
 1.219  24-Nov-2020  rillig make(1): indent make.h with tabs instead of spaces
 1.218  24-Nov-2020  rillig make(1): add high-level API for GNode.made

Having an enum whose constants must be ordered in a certain way may be
unexpected to casual readers. Hide this implementation detail in
separate functions.
 1.217  24-Nov-2020  rillig make(1): fix error handling for dependency of .END in -k mode

Fix one bug, find 4 new ones. All these bugs have been around since
2005-05-08, when dependencies on the .BEGIN, .END and .INTERRUPT nodes
were implemented. Before that, checking gn->made == ERROR was
appropriate, but adding the dependencies made ABORTED a new possible
error value from Compat_Make.
 1.216  24-Nov-2020  rillig make(1): document the enum GNodeMade

Given only the state names and their individual documentation, it is
hard to see the full picture. To make this easier, provide typical
examples of the ways that a GNode takes through these states.
 1.215  23-Nov-2020  rillig make(1): align end-of-line comments with tabs
 1.214  22-Nov-2020  rillig make(1): add more debugging for searching transformation rules
 1.213  22-Nov-2020  rillig make(1): renumber the flags in GNodeType

In plain C, bit 31 of an enum is not guaranteed to play well with binary
operators. Bit 23 was unused until now, so shift them all down.

The order of the bits is only used in Targ_PrintType.
 1.212  21-Nov-2020  rillig make(1): rename type Suff to Suffix

This avoids confusion with the module suff.c of the same name.

The existing naming convention is that the module's public functions are
named Suff_Func and its private functions are named SuffFunc. This
collides with the naming convention that a the functions of the type Suff
are called Suff_Func as well.

To resolve this collision, rename Suff to Suffix. The related types like
SuffList and the affected functions like SuffRef will be renamed in a
follow-up commit.
 1.211  21-Nov-2020  rillig make(1): rename GNode.context to vars

Especially in the variables module, the expression 'ctxt->context' looked
redundant. Having a GNode as a context sounds ok, but a context of a
context just doesn't make sense.
 1.210  16-Nov-2020  rillig make(1): clean up and extend comments in targ.c
 1.209  15-Nov-2020  rillig make(1): clean up make.h, meta.c, parse.c, str.c

The main changes are in the comments, which have been shortened and
corrected.

Some local variables changed their names.

In ParseErrorInternal, the scope of va_start is now narrower.

In ParseDoDependency, the type of tOp has been fixed.

ParseGetLine doesn't take flags anymore but instead a parsing mode.
Previously, the flags had not been combined anyway.

At the beginning of Parse_File, fatals is already guaranteed to be 0, and
even if not, it would be wrong to just discard the fatal errors.
 1.208  14-Nov-2020  rillig make(1): remove redundant parameter from str2Lst_Append
 1.207  14-Nov-2020  rillig make(1): rename DEFAULT to defaultNode

In C, uppercase names are typically used for constants, and this is not
a constant.
 1.206  12-Nov-2020  sjg Pass a writable flag to Main_SetObjdir to control writable check

For curdir and an explicit .OBJDIR target, we allow for
the directory to be read-only.
During InitObjdir we otherwise default to requiring objdir to be
writable - this can be controlled by env variable
MAKE_OBJDIR_CHECK_WRITABLE

Add unit-tests/objdir-writable

Reviewed by: christos rillig
 1.205  10-Nov-2020  rillig make(1): use consistent definition for MAKE_INLINE
 1.204  08-Nov-2020  rillig make(1): move lint option away from the debug flags

The lint option affects much more than a typical debug flag. Not only
does it produce additional diagnostics, it also changes the control flow
and fixes the traditional error handling in a few places.

No functional change.
 1.203  08-Nov-2020  rillig make(1): sort debug flags alphabetically

DEBUG_FOR had been accidentally left at the end of the list.
 1.202  08-Nov-2020  rillig make(1): fix documentation of preserveUndefined

Since it is a global variable and not a VarEvalFlags, it affects nested
subexpressions as well, and there is no way to escape it.
 1.201  08-Nov-2020  rillig make(1): fix debug output in out-of-date for cohorts (since 2003-11-14)

Before, a wrong cause for being out-of-date was printed in the debug log,
for optional cohorts. This was caused by having the same conditions
duplicated in the code, instead of putting them in a separate function.

Now the optional cohort is correctly identified as using the '::'
dependency operator.
 1.200  08-Nov-2020  rillig make(1): rename Make_OODate to GNode_IsOODate

It doesn't matter which of the make modules is in charge of determining
whether a node is out-of-date. Therefore, remove the module name from
the function name.
 1.199  08-Nov-2020  rillig make(1): rename Make_TimeStamp to GNode_UpdateYoungestChild
 1.198  08-Nov-2020  rillig make(1): fix type mismatch between int and PrintVarsMode
 1.197  07-Nov-2020  rillig make(1): add pp_skip_hspace to skip horizontal whitespace during parsing
 1.196  06-Nov-2020  rillig make(1): merge duplicate code for skipping horizontal whitespace
 1.195  06-Nov-2020  rillig make(1): move name of inline functions to the beginning of the line
 1.194  06-Nov-2020  rillig make(1): rename dieQuietly to shouldDieQuietly

It was too confusing to have a function named die that doesn't actually
die. Plus, the return type int didn't give any clue about what the
function actually returns.
 1.193  06-Nov-2020  rillig make(1): document DEBUG_LINT
 1.192  06-Nov-2020  rillig make(1): sort and renumber debug flags
 1.191  06-Nov-2020  rillig make(1): document progname
 1.190  06-Nov-2020  rillig make(1): clean up comments in make.h
 1.189  05-Nov-2020  rillig make(1): extend documentation for GNodeType constants
 1.188  04-Nov-2020  rillig make(1): group GNode fields by topic, document ParseErrorLevel
 1.187  04-Nov-2020  rillig make(1): align comments on a few commonly used variables
 1.186  04-Nov-2020  rillig make(1): move declaration of parseIncPath from main.c to make.h
 1.185  04-Nov-2020  rillig make(1): rename MAKEFILE_PREFERENCE for consistency

The names of the other special variables are all prefixed with MAKE_.
 1.184  04-Nov-2020  rillig make(1): remove unused names DTARGET, FPREFIX, etc.
 1.183  04-Nov-2020  rillig make(1): negate discardUndefined to preserveUndefined
 1.182  04-Nov-2020  rillig make(1): rename oldVars to discardUndefined

While here, moved all the documentation about this variable into a
single place.
 1.181  04-Nov-2020  rillig make(1): add missing enum tag for GNodeMade
 1.180  04-Nov-2020  rillig make(1): document that "old-style" variables are older than 1993
 1.179  01-Nov-2020  rillig make(1): negate NoExecute to GNode_ShouldExecute
 1.178  31-Oct-2020  rillig make(1): do not look up local variables like .TARGET anywhere else

Nobody defines a global variable named .TARGET since that would have
many unpredictable effects, applying to all targets at once.

Nobody defines an environment variable named .TARGET since that's
against the naming conventions for environment variables and would have
the same effect.

Because of this, there is no point looking up the variables that are
local to a GNode anywhere else. This means they cannot come from the
environment and thus their value doesn't need to be freed after use,
which makes the code simpler.

The newly added accessor functions in make.h refer to external
functions, but since that header is not used anywhere outside of
usr.bin/make, it doesn't matter. Between 2020-08-25 and 2020-10-30,
that header had been referenced by usr.bin/xinstall.
 1.177  30-Oct-2020  rillig make(1): clean up comments and local variables in var.c
 1.176  30-Oct-2020  rillig make(1): rename VAR_CMD to VAR_CMDLINE

Since make has to do with both the command line and child commands, the
former name was confusing.
 1.175  28-Oct-2020  rillig make(1): rename defIncPath to defSysIncPath

There are two variables, parseIncPath and sysIncPath, which made the
name defIncPath ambiguous.
 1.174  27-Oct-2020  rillig make(1): move debugVflag to CmdOpts
 1.173  26-Oct-2020  rillig make(1): group the command line options and arguments

By having a single struct that holds all command line options and
arguments, it is easy to see in the code when such a command line
argument is modified. It also cleans up the namespace since the command
line options don't follow a common naming style. Having them in a
struct also means that there is a single place for putting the
documentation, not two as before.

The struct also suggests to extract the initialization code out of main,
which is still too large, having more than 400 lines of code and
covering far too many topics.
 1.172  25-Oct-2020  rillig make(1): add GNode_Path to access the path of a GNode
 1.171  24-Oct-2020  rillig make(1): remove macros MIN and MAX

These macros typically evaluate one of their arguments twice. Until
2020-08-31, they had not parenthesized their arguments properly. They
are only used in a few places, therefore it doesn't hurt much to have
them expanded.
 1.170  24-Oct-2020  rillig make(1): convert macros for debug flags into enum
 1.169  24-Oct-2020  rillig make(1): remove lengthy comments from archive handling

The relevant information is already documented on OP_ARCHV, but only
since a few weeks.
 1.168  23-Oct-2020  rillig make(1): fix typos in comments
 1.167  23-Oct-2020  rillig make(1): convert error level for Parse_Error to an enum
 1.166  23-Oct-2020  rillig make(1): rename GNode.cmgn to youngestChild

The name is longer than before but also clearer.
 1.165  23-Oct-2020  rillig make(1): add test for the '::' dependency operator
 1.164  23-Oct-2020  rillig make(1): negate OP_NOP and rename it to GNode_IsTarget
 1.163  23-Oct-2020  rillig make(1): merge duplicate documentation for OP_NOP
 1.162  23-Oct-2020  rillig make(1): only define TRUE and FALSE if necessary

For -DUSE_ENUM_BOOLEAN, the macros are not necessary.
 1.161  23-Oct-2020  rillig make(1): allow compilation with Boolean implemented as char
 1.160  19-Oct-2020  rillig make(1): clean up headers, no functional change
 1.159  18-Oct-2020  rillig make(1): add tags to enum types

This allows IDEs to offer better type information than "anonymous enum".
 1.158  18-Oct-2020  rillig make(1): remove underscore from Hash_Table and Hash_Entry

For consistency with the other type names, such as GNodeListNode.
 1.157  18-Oct-2020  rillig make(1): convert NoExecute to non-inline

The header make.h is not only used by usr.bin/make but also by
usr.bin/xinstall, to get the needshell function.

That program does not declare the variables noRecursiveExecute and
noExecute, leading to an "unresolved reference" at link time. To fix
this, move the inline functions away from make.h.

It's a quirk that make.h gets included by other programs since it
contains a whole lot of definitions that are only useful in make itself.
If any, there should be a separate header that defines the public
library API of make, and only that.
 1.156  05-Oct-2020  rillig make(1): enable format string truncation warnings for parse.c

No node is ever going to have a million or more cohorts. And even if
there were, this name is only used for printing informational messages.
 1.155  05-Oct-2020  rillig make(1): revert previous commit

It had accidentally reverted all the work from the past few days.
 1.154  05-Oct-2020  rillig make(1): fix double-free bug in -DCLEANUP mode (since 2020-10-02)

The bug had been introduced with dir.c 1.155 on 2020-10-02 22:20:25. In
that commit, openDirectories was replaced with a combination of a list
with a hash table, for more efficient lookup by name.

Upon cleanup, OpenDirs_Done is called, which in turn called
Dir_ClearPath. Dir_ClearPath takes full ownership of the given list and
empties it. This was no problem before since afterwards the list was
empty and calling Lst_Free just frees the remaining list pointer.

With OpenDirs, this list was combined with a hash table, and the hash
table contains the list nodes, assuming that the OpenDirs functions have
full ownership of both the list and the hash table. This assumption was
generally correct, except for the one moment during cleanup where full
ownership of the list was passed to Dir_ClearPath, while the hash table
still contained pointers to the (now freed) list nodes. This by itself
was not a problem since the hash table would be freed afterwards. But
as part of Dir_ClearPath, OpenDirs_Remove was called, which looked up
the freed directory by name and now found the freed list node, trying to
free it again. Boom.

Fixed by replacing the call to Dir_ClearPath with code that only frees
the directories, without giving up control over the list.
 1.153  05-Oct-2020  rillig make(1): replace GNode macros with inline functions

Curiously, this affects the size of the generated code: it gets smaller
than before.
 1.152  03-Oct-2020  rillig make(1): use consistent pattern for parsing whitespace

The pp and cpp in the function names stand for "parsing position" and
"const parsing position".
 1.151  28-Sep-2020  rillig make(1): remove the inline attribute from debug_printf

The code size overhead of a varargs function is much more than I had
expected. Since GCC 5 didn't inline the function anyway and it is only
used in debug mode, there is no point keeping multiple copies of that
code around.
 1.150  28-Sep-2020  rillig make(1): make debug logging simpler

This avoids referring to the debug_file variable in many places where
this implementation detail is not necessary.
 1.149  28-Sep-2020  rillig make(1): make debugging code shorter
 1.148  27-Sep-2020  rillig make(1): normalize whitespace in source code

There is no more space tab. Either only tabs or only spaces or tabs
followed by spaces, but not spaces followed by tabs.
 1.147  26-Sep-2020  rillig make(1): clean up API for finding and creating GNodes

The previous API had complicated rules for the cases in which the single
function returned NULL or what it did. The flags for that function were
confusing since passing TARG_NOHASH would create a new node even though
TARG_CREATE was not included in that bit mask.

Splitting the function into 3 separate functions avoids this confusion.
It also reveals several places where the complicated API led to
unreachable code. Such code has been removed.
 1.146  24-Sep-2020  rillig make(1): migrate MakeTimeStamp and MakeUnmark to Lst_ForEach
 1.145  23-Sep-2020  rillig make(1): fix assertion failure in -j mode with .END node

There had been two separate global variables for the .END node, and in
parallel mode, only the one in jobs.c was initialized.

The code in JobRun heads over to Compat_Make without calling Compat_Run
first, which left the variable ENDNode uninitialized.
 1.144  22-Sep-2020  rillig make(1): use fine-grained type names for lists and their nodes

This is only intended to help the human reader. There is no additional
type safety yet.
 1.143  21-Sep-2020  rillig make(1): add specific typedefs for lists

These typedefs are only intended to help human readers, they do not
provide any type-safety. They also make the pointers explicit, which
had been hidden before by the typedef for Lst and LstNode. Typing a few
'*' is less work than finding out which of the many types are pointers
and which aren't.

In meta.c, the variable "ln" served two completely different purposes,
which have been split again. Register allocation is the job of the
compiler, not of the human source code reader.
 1.142  13-Sep-2020  rillig make(1): clean up RCSID blocks

These blocks mostly consisted of redundant structure, following the same
#ifndef pattern over and over, with only minimal variation.

It's easier to maintain if the common structure is only written once and
encapsulated in a macro.

To avoid "defined but unused" warnings from GCC in the case where
MAKE_NATIVE is not defined, I had to add volatile. Adding
MAKE_ATTR_UNUSED alone would not preserve the rcsid variable in the
resulting binary.
 1.141  13-Sep-2020  rillig make(1): clean up API for evaluating conditions

There was no need to make struct If publicly visible.

There was no need to have parameters in the public API that were passed
the same constants all the time.

The former function names had not been distinctive.
 1.140  13-Sep-2020  rillig make(1): remove #endif comments for very short blocks
 1.139  11-Sep-2020  rillig make(1): add wrappers around ctype.h functions

This avoids casting the argument to unsigned char, and to cast the
result of toupper/tolower back to char.
 1.138  11-Sep-2020  rillig make(1): add missing space in source code
 1.137  02-Sep-2020  rillig make(1): document the value restrictions for Boolean variables

The previous lenient rule came from the sprite.h header that was not
specific to make. To avoid confusion, only the expected values should
be stored in a Boolean variable. To help find obvious violations and
inconsistencies, there are different possibilities for the Boolean type,
during development.

In C there is no way to actually enforce this restriction at runtime.
It would be possible in C++, but the code is not ready to be compiled
with a C++ compiler.
 1.136  02-Sep-2020  rillig make(1): reduce number of stat fields returned by cached_stat

Only st_mtime and st_mode are actually filled, the remaining fields had
been set to zero. To prevent these from ever being accessed, a custom
struct make_stat replaces the previously used struct stat.

The fields in struct make_stat are intentionally named different from
the fields in struct stat because NetBSD and some other operating
systems define st_mtime as a macro, and that would not work in a field
declaration.
 1.135  02-Sep-2020  rillig make(1): use proper types in API of cached_stat and cached_lstat
 1.134  31-Aug-2020  rillig make(1): parenthesize macro arguments

Just in case anyone wants to use them for copy-and-paste.

The invocations of these macros are left cautious since the
system-provided definition of these macros may have forgotten the
parentheses as well.
 1.133  30-Aug-2020  rillig make(1): rename GNode.iParents to implicitParents

The i alone was too ambiguous. It could have meant ignore, implicit,
interactive, and probably many more.
 1.132  29-Aug-2020  rillig make(1): remove ReturnStatus, SUCCESS and FAILURE

These are used in so few places now that it is easier to use a simple
Boolean for them.
 1.131  29-Aug-2020  rillig make(1): add another Boolean variant to check during development
 1.130  29-Aug-2020  rillig make(1): allow for strict type checking for Boolean

Having Boolean aliased to int creates ambiguities since int is widely
used. Allow to occasionally compile make with -DUSE_DOUBLE_BOOLEAN to
check that the type definitions still agree.
 1.129  28-Aug-2020  rillig make(1): clean up targ.c

The main part is removing redundant or outdated information from
comments. In a few places, the expression cond ? TRUE : FALSE has been
simplified.
 1.128  28-Aug-2020  rillig make(1): print suffix flags in the standard way

This changes the output (it is now SUFF_NULL instead of just NULL), and
the order of the flags in the output is reversed.
 1.127  26-Aug-2020  rillig make(1): remove header sprite.h

Make is independent of the Sprite operating system.
 1.126  24-Aug-2020  rillig make(1): in debug mode, print GNode details in symbols

A string like OP_DEPENDS|OP_OPTIONAL|OP_PRECIOUS is much easier to read
and understand than the bit pattern 00000089.

The implementation in enum.h looks really bloated and ugly, but using
this API is as simple and natural as possible. That's the trade-off.

In enum.h, I thought about choosing the numbers in the macros such that
it is always possible to combine two of them in order to reach an
arbitrary number, because of the "part1, part2" in the ENUM__SPEC macro.
The powers of 2 are not these numbers, as 7 cannot be expressed as the
sum of two of them. Neither are the fibonacci numbers since 12 cannot
be expressed as the sum of 2 fibonacci numbers. I tried to find a
general pattern to generate these minimal 2-sum numbers, but failed.
 1.125  23-Aug-2020  rillig make(1): remove unused declarations from header files
 1.124  23-Aug-2020  rillig make(1): use proper data type for GNode.type

Theoretically this should show the enum constant names when displaying
the field in the debugger. This only happens for GNode.flags though,
but not for GNode.type. Not sure what the exact cause is.
 1.123  22-Aug-2020  rillig make(1): clean up headers

Remove redundant headers that are already included by "make.h".
Make <assert.h> available to all compilation units that use "make.h".
 1.122  22-Aug-2020  rillig make(1): remove wrong comment about possible Y2038 problem

The "time" that is mentioned is not clock time but a sequence number.
 1.121  22-Aug-2020  rillig make(1): remove unused variable jobServer
 1.120  22-Aug-2020  rillig make(1): convert GNode type constants to enum

This allows debuggers to print symbolic names for the type flags.

The names of these constants need to be revised. These flags probably
started as the 3 dependency operators (:, !, ::), and at that time, the
prefix OP_ made sense. The other flags are not related to the type of
the dependency operator and could have been placed in GNode.flags as
well.
 1.119  22-Aug-2020  rillig make(1): restructure GNode types and documentation

Having a numbered list above the type was not helpful since the numbers
didn't serve any purpose, they just consumed screen space.

Several of these list items didn't have an obvious relationship to the
struct fields. It's better to have just a rough introduction as the
type level documentation, leaving the details to the individual fields.

Converting the types and flags and other constants into separate types
and defining them outside the struct leaves more space to see the
relationship of the struct fields.

Limiting the documentation of each field to a single line, as suggested
by the end-of-line comments, reduces clarity since several of the fields
need way more documentation to be properly understood.
 1.118  20-Aug-2020  rillig make(1): remove unused function declarations
 1.117  20-Aug-2020  rillig make(1): fix wrong or outdated comments
 1.116  13-Aug-2020  rillig make(1): follow naming conventions for multiple-inclusion guards

This avoids undefined behavior.
 1.115  11-Aug-2020  rillig make(1): convert Suff.flags from #define to enum

This increases debugging support since the debugger can now display
symbolic names instead of an integer bit mask.
 1.114  10-Aug-2020  rillig make(1): fix parameter name of str_concat

The previous documentation mentioned Str_Concat, but str_concat has been
written in lowercase for years. The "flags" are not flags since they
cannot be combined, not even when they are written in hex.
 1.113  01-Aug-2020  rillig make(1): use enum for return values of Cond_Eval and friends
 1.112  31-Jul-2020  sjg Add -dL for LINT

When parsing variable assignments other than := and if
value contains '$' attempt Var_Subst the same as for :=,
if the value does not parse correctly, we get a fatal error
including file an line number.

This can greatly help with finding the cause of problems.

Reviewed by: christos
 1.111  21-Jul-2020  rillig make(1): clean up various things in var.c
 1.110  18-Jul-2020  sjg Add -dh for DEBUG_HASH

Allow tracking of max chain length, to see how well the hash
tables are working.
Pull the actual hash operation into a marco so it can be
easily changed - for experimenting.

The current hash, is pretty good.

Reviewed by: christos
 1.109  02-Jul-2020  rillig make(1): add more type safety for enums

There are several types of flags for variables, and these cannot be
mixed. To prevent accidental typos, these are defined in separate enum
types. Clang warns about direct assignments between distinct types, but
not about mixing distinct types in binary expressions like A | B. GCC
does not warn at all.
 1.108  19-Jun-2020  sjg Avoid unnecessary noise when sub-make or sibling dies

When analyzing a build log, the first 'stopped' output
from make, is the end of interesting output.

Normally when a build fails deep down in a parallel build
the log ends with many blockes of error output from make,
with all but the fist being unhelpful.

We add a function dieQuietly() which will return true
if we should supress the error output from make.
If the failing node was a sub-make, we want to die quietly.

Also when we read an abort token we call dieQuietly telling we
want to die quietly.

This behavior is suppressed by -dj or
setting .MAKE.DIE_QUIETLY=no

Reviewed by: christos
 1.107  03-Apr-2020  sjg Revert arraycount
 1.106  03-Apr-2020  sjg CHECK_VALID_META is too agressive for CMD check.

An empty CMD is perfectly valid.
So using CHECK_VALID_META results in too much rebuilding.
 1.105  30-Mar-2020  sjg make: fix -fno-common build

debug was declared extern, but debug_file was not; correct this and define
debug_file in main.c (as debug is) to fix the -fno-common build.

-fno-common will become the default with GCC10/LLVM11.

Patch from kevans at freebsd
 1.104  12-Feb-2018  sjg branches: 1.104.4; 1.104.6;
Do not treat .info as warning for -W

Reported by: lwhsu at FreeBSD.org
 1.103  20-Jul-2017  sjg Make compat.c handle SIGINT etc more like job.c

If there is a running child, pass the signal on, and
wait for it to exit before we self-terminate.

Reviewed by: christos
 1.102  07-Dec-2016  christos branches: 1.102.6;
Refactor and simplify objdir setting code.
 1.101  26-Aug-2016  dholland Add a .DELETE_ON_ERROR: magic target that causes *failed* targets as
well as *interrupted* targets to be deleted. The name and behavior of
the variable matches gmake.

Also fix a glitch in newline output on error in compat mode that I
discovered while doing it.

Closes PR 51376.
 1.100  07-Jun-2016  sjg branches: 1.100.2;
Extend the mtimes cache used by dir.c so it can be used by others.

We store both st_mtime and st_mode, since some callers care about the
later.

Reviewed by: christos
 1.99  03-Jun-2016  sjg Add cached_realpath()

realpath(3) is expensive, and meta mode at least uses it extensively.
We use cached_realpath() to save the result of successful calls to
realpath(3) in a private variable context.

This improves the worst case performance (eg examining libc with
nothing to do) by a factor of 4.

Reviewed by: christos
 1.98  18-Feb-2016  christos Collapse the 3 boolean parameter to 1 flags parameter. No functional change.
 1.97  17-Jan-2016  christos provide missing FD_CLOEXEC for the havenots.
 1.96  21-Sep-2015  pooka Add more compat defs for code compiled for !NetBSD targets.

At least GNU Hurd lacks MAXHOSTNAMELEN/MAXPATHLEN/PATH_MAX, so make sure
those have some arbitrary semi-sane values.

from Robert Millan in PR misc/50166, with some modifications from myself
 1.95  07-Sep-2014  joerg Revert all make changes except the unit tests to the state of three
weeks ago. Individual changes can be reapplied after review.
 1.94  23-Aug-2014  christos PR/46096: Jarmo Jaakkola: fix many problems with dependencies (PR 49086)

Quite extensive rewrite of the Suff module. Some ripple effects into
Parse and Targ modules too.

Dependency searches in general were made to honor explicit rules so
implicit and explicit sources are no longer applied on targets that
do not invoke a transformation rule.

Archive member dependency search was rewritten. Explicit rules now
work properly and $(.TARGET) is set correctly. POSIX semantics for
lib(member.o) and .s1.a rules are supported.

.SUFFIXES list maintenance was rewritten so that scanning of existing
rules works when suffixes are added and that clearing the suffix list
removes single suffix rules too. Transformation rule nodes are now
mixed with regular nodes so they are available as regular targets too
if needed (especially after the known suffixes are cleared).

The .NULL target was documented in the manual page, especially to
warn against using it when a single suffix rule would work.
A deprecation warning was also added to the manual and make also
warns the user if it encounters .NULL.

Search for suffix rules no longer allows the explicit dependencies
to override the selected transformation rule. A check is made in
the search that the transformation that would be tried does not
already exist in the chain. This prevents getting stuck in an infinite
loop under specific circumstances. Local variables are now set
before node's children are expanded so dynamic sources work in
multi-stage transformations. Make_HandleUse() no longer expands
the added children for transformation nodes, preventing triple
expansion and allowing the Suff module to properly postpone their
expansion until proper values are set for the local variables.

Directory prefix is no longer removed from $(.PREFIX) if the target
is found via directory search.

The last rule defined is now used instead of the first one (POSIX
requirement) in case a rule is defined multiple times. Everything
defined in the first instance is undone, but things added "globally"
are honored. To implement this, each node tracks attribute bits
which have been set by special targets (global) instead of special
sources (local). They also track dependencies that were added by
a rule with commands (local) instead of rule with no commands (global).

New attribute, OP_FROM_SYS_MK is introduced. It is set on all targets
found in system makefiles so that they are not eligible to become
the main target. We cannot just set OP_NOTMAIN because it is one of
the attributes inherited from transformation and .USE rules and would
make any eligible target that uses a built-in inference rule ineligible.

The $(.IMPSRC) local variable now works like in gmake: it is set to
the first prerequisite for explicit rules. For implicit rules it
is still the implied source.

The manual page is improved regarding the fixed features. Test cases
for the fixed problems are added.

Other improvements in the Suff module include:
- better debug messages for transformation rule search (length of
the chain is now visualized by indentation)
- Suff structures are created, destroyed and moved around by a set
of maintenance functions so their reference counts are easier
to track (this also gets rid of a lot of code duplication)
- some unreasonably long functions were split into smaller ones
- many local variables had their names changed to describe their
purpose instead of their type
 1.93  16-Jul-2014  christos PR/48367: David A. Holland: Mark possible submake nodes so that we can
avoid closing the job pipe on exec for them in order to make recursive
makes work in parallel.
 1.92  04-Sep-2013  sjg branches: 1.92.2;
Add VAR_INTERNAL as a context for variables set by make itself,
which should not override those set by makefiles.
Currently MAKEFILE is the only variable affected.

Reviewed by: christos
 1.91  18-Jun-2013  sjg Use a #define for the variable name we put in environment to pass .MAKE.LEVEL
in case we don't want to use gmake's MAKELEVEL in a different way.
 1.90  25-Feb-2013  dholland Fix obvious typo.
 1.89  12-Jun-2012  joerg branches: 1.89.2;
Replace __dead, __unused and the various printf format attributes
with versions prefixed by MAKE_ATTR_* to avoid modifying the
implementation namespace. Make sure they are available in all places
using nonints.h to fix bootstrap on Linux.
 1.88  04-Jun-2012  sjg Var_UnExport() and setenv() can both realloc environ.
Use a common variable (savedEnv) to track that to avoid wasting memory.
Also, if providing setenv and unsetenv, do getenv too to ensure a consistent
set.
 1.87  16-Sep-2011  joerg branches: 1.87.2;
Use __dead consistently. If it doesn't exist, define it away.
 1.86  04-May-2011  sjg Add .MAKE.META.BAILIWICK - to identify the scope of make's control.
meta_oodate: if a file that was written or linked within our bailiwick,
but outside of .OBJDIR is missing, add it to missingFiles.
If we get to the end of the .meta file without seeing it [re]moved,
then consider the target out-of-date.
 1.85  20-Feb-2011  joerg Use the current directory at start time for .PARSEDIR, if the filename
contains no /.
 1.84  25-Nov-2010  christos branches: 1.84.2;
Instead of keeping around the mtime of the youngest child, keep a pointer
to it, so that we can print it when we do the out of date determination.
 1.83  13-Sep-2010  sjg Add meta.c which implements "meta" mode for make.
In this mode, a .meta file is created for each target, capturing
the expanded commands used, any command output, and if filemon(9)
is available, a record of system calls which are of interest.
Not enabled unless USE_META=yes is set when building make.
Also, if FILEMON_H exists, meta.c will be compiled to use filemon(9).
 1.82  23-Apr-2010  sjg On darwin at least, vfork() fails in child of vfork().
It probably shouldn't work anyway, so avoid this.
We use the macro vFork() - a function seems to cause problems
and is unnecessary overhead.
 1.81  22-Apr-2010  sjg Just because $TMPDIR is set does not mean it is valid.
Add a central function for creating temp files so we have one place to
audit.

Reviewed by: dh
 1.80  07-Apr-2010  sjg Add:

.error "message"
.warning "message"
based on FreeBSD implementation.
add .info while were at it.

.ERROR: a target to run on error.
We pass the failing GNode to PrintOnError so it can set
.ERROR_TARGET.

.MAKE.MAKEFILE_PREFERENCE
As a means to control make's list of prefered makefile names.
(Default: "makefile Makefile")

.MAKE.DEPENDFILE
Names the file to read dependencies from
(Default ".depend")

.MAKE.MODE
Processed after all makefiles are read.
Can put make into "compat" mode (more to come).

Fix:

compat.c: Error code should not be sent to debug_file.
Make_DoAllVar: use DONE_ALLSRC to avoid processing a node multiple times.
ReadMakefile: we can simply use doing_depend to control setting MAKEFILE.
 1.79  08-Sep-2009  sjg Reviewed by: apb

Use .MAKE.LEVEL to track recursion.
The first instance of make will have .MAKE.LEVEL 0, which
can be handy for excluding rules which should not apply
in a sub-make.
gmake and freebsd's make have a similar mechanism, but each
uses a different variable to track it. Since we cannot be
compatible with both, we allow the makefiles to cope if they want
by handling the export of .MAKE.LEVEL+1 in Var_Set().
 1.78  24-Jan-2009  dsl Fixes to includes of make_malloc.h to that it actually builds when
USE_EMALLOC is undefined.
Fixes earlier fixes :-)
 1.77  24-Jan-2009  dsl Move the bmake_malloc() functions into their own .c and .h files.
Include instead of make.h in a few places.
 1.76  13-Dec-2008  dsl Use NULL instead of -1 cast to the relavant type (usually via NIL).
This was a suggestion from christos - so blame him if there is a deep
reason for using -1 :-)
 1.75  07-Dec-2008  christos add debug flag for cwd.
 1.74  15-Feb-2008  christos back all changes out until I fix it properly.
 1.73  14-Feb-2008  christos - use pid_t/size_t as appropriate instead of int.
- use %ld to print pids.
- fix a bit of lint.
- WARNS=4
 1.72  19-Jan-2008  sjg Implement -dl (aka LOUD) to override '@' at the start of script lines.
Based on supplied patch.

PR: 37202
 1.71  21-Dec-2007  dsl If make can't find a rule to create a file it needs to satisfy a rule
that came from .depend then just output a message and contine.
 1.70  08-Oct-2007  sjg Use .MAKE.MAKEFILES to track all the makefiles that have been read
so they can be used in dependency rules.
 1.69  05-Oct-2007  sjg Add the ability to .export variables to the environment.
 1.68  01-Oct-2007  sjg Add the ability to tweak the token output before targets in job mode.
Eg.
.MAKE.JOB.PREFIX=${.newline}---${.MAKE:T}[${.MAKE.PID}]
would produce
---make[1234] target ---
 1.67  01-Jan-2007  dsl branches: 1.67.4;
Remove the (now redundant) MAKE_BSIZE, BUF_DEF_SIZE from buf.c is used instead.
 1.66  20-Dec-2006  dsl Fix detection and reporting of dependency loops in parallel makes.
make's unit_tests should now run again.
 1.65  04-Dec-2006  dsl There really is no point calling estrdup() to get a copy of the makefile name
into every 'gnode' (aka target), instead just copy a pointer to the string
and avoid freeing the original name when we close the file.
I can't imagine any makefile set where this gives a larger footprint!
 1.64  17-Nov-2006  dsl branches: 1.64.2;
A rather large rototil in the way the parallel make code schedules jobs.
This gives a considerable speedup in the processing of .WAIT and .ORDER.
Both .WAIT and .ORDER stop both the commands of the node, and its dependant
nodes being built until the LH nodes are complete.
.WAIT only applies to the dependency line on which it appears, whereas
.ORDER applies globally between the two nodes.
In both cases dependant nodes can be built because other targets need them.
make now processes the target list left to right, scheduling child nodes
as they are needed to make other nodes (instead of attempting to generate
a bottom-up dependency graph at the start). This means that 'make -j1'
will tend to build in the same order as a non-parallel make.
Note that:
all: x y
x: a .WAIT b
y: b .WAIT a
does not generate a dependency loop.
But
x: y
.ORDER y x
does (unless something elswhere causes 'y' to be built).
 1.63  15-Oct-2006  dsl Output all debug trace output through 'debug_file' defaulting to 'stdout'.
(Almost all the debug output went there, but some went to stderr.)
Split the parsing of -d (debug flags) out into its own routine.
Allow the output filename to be changed by specifying -dF<file> to create
a log file, or -dF+<file> to append to it. <file> may be stdout or stderr.
Also change so that -d-<flags> acts on <flags> locally but doesn't copy
them to MAKEFLAGS so they aren't inherited by child makes.
I'm not 100% happy with the command line syntax for the above, so they are
currently undocumented.
 1.62  09-Oct-2006  dsl Rip out the code for the undocumented -P (don't use pipes for command
output) option. I'm sure it is baggage from the past.
 1.61  26-Aug-2006  christos Don't include <util.h> if we are not NetBSD. Other OS's don't have it.
 1.60  26-Aug-2006  christos More programs using efun.
 1.59  10-Mar-2006  christos PR/33049: OKANO Takayoshi: Fix the spelling of NetBSD RCS tags.
 1.58  26-Feb-2006  apb Make ".WAIT" apply recursively to all children of nodes on the right
hand side of the .WAIT, except when the recursive interpretation would
cause a cycle in the dependency graph.

Discussed in tech-toolchain. Reviewed by christos, sjg.
 1.57  22-Jan-2006  dsl Make .NOTPARALLEL stop the job engine issuing more than one token (instead
of trying to de-jobify the make.
You can now put .NOTPARALLEL in a submake of a recursive make (where it is
using a job-token pipe from the outer make and have it only run a single job.
You can also specify .NOTPARRALEL in the root makefile of a large recursive
make and have the submakes run multiple commands.
Add some diagnostics printfs (enabled with -dp) to the parser.
 1.56  24-Dec-2005  perry Remove leading __ from __(const|inline|signed|volatile) -- it is obsolete.
 1.55  17-Jun-2005  jmc Add new debugging option '-dn' which will leave the scripts fed into make on
-j jobs in /tmp for debugging purposes. Add a note to the man page that
this could cause problems if run a lot (due to the number of files created)
 1.54  08-May-2005  christos Now that dependencies in .BEGIN, .END, and .INTERRUPT work, allow them.
 1.53  01-May-2005  christos Don't accept dependencies for .BEGIN, .END, and .INTERRUPT since they
don't make sense or work anyway.
 1.52  15-Jul-2004  yamt define __GNUC_PREREQ__ if it isn't available on host.
 1.51  01-Jul-2004  jmc Change to use __unused instead and provide a compat definition in make.h if
not already defined from cdefs.h
 1.50  07-Dec-2003  dsl Add a -Dg3 which outputs the 'input graph' only on error exit.
Lets you see the wood for the trees...
 1.49  27-Sep-2003  sjg Implement :[] modifier to allow picking a range of words out of a variable.
Also :tW and a W flag to :C and :S to allow treating value as a single word.
Add unit tests for the above, and fix some corner cases.
Based on patches supplied by Alan Barrett <apb@cequrux.com>
 1.48  10-Sep-2003  jmmv Add the 'e' debug flag (i.e., '-d e'): when enabled, show the "target
failed" and "command failed" messages added recently. These introduce
too much noise when debugging some kind of problems, specially in pkgsrc.
 1.47  07-Aug-2003  agc Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22365, verified by myself.
 1.46  14-Jul-2003  christos Pass WARNS=3
 1.45  14-Mar-2003  thorpej Add a -X option, which prevents make(1) from putting variables
set on the command line into the environment individually. This
can be useful on systems which have a tight limit on the size
of the argument space.
 1.44  15-Jun-2002  wiz Remove !__STDC__ stuff, de-__P(), ANSIfy, and de-register.
 1.43  20-Mar-2002  pk Xref: parse.c, rev 1.46; make.c, rev 1.23

In these revisions `::' dependency handling was simplified by not linking
the cohort nodes into the dependency graph. This broke dependency checking
on all but the first instance of a `::' target since all of the cohort nodes
now just form a collection of disconnected dependency graphs.

Fix this by keeping a back-reference in each cohort to its leader (the
first instance of a :: node with the same name) and a count of the number
of cohorts that need to be made before dependent nodes are scheduled.

Classically, we'd need six centurions for cohort, but in this case one
suffices...
 1.42  18-Feb-2002  pk Make_HandleUse() no longer has a return value,
 1.41  04-Feb-2002  christos Remove OP_NOSUFF, fix OP_MADE in compat mode [from pk], and make the OP_MADE
comment reflect reality.
 1.40  03-Feb-2002  christos Bring down the number of stat(2) system calls from 682 to 294. This change
adds a .NOSUFF directive that has been applied to targets that have been
already made, and are not supposed to have suffix rules applied to them.
 1.39  31-Oct-2001  tv Overhaul the initialization and handling of .OBJDIR:

* Replace chdir_verify_path() with Main_SetObjdir(), which can be called
externally, and can take a "const char *". (There's a lot of non-const
"char *" passing around in var.c of what should be const strings....)

* Rewrite the initial "find my .OBJDIR" code to make use of the new
function. This still functions as it had in the past, but the comment
above this block was changed to reflect reality: if MAKEOBJDIRPREFIX
or MAKEOBJDIR are set in the environment, then *only that value* is
tried; make does not fall back to obj.MACHINE, obj, and /usr/obj/`pwd`
as it would without these env vars set.

* Add a new special target, .OBJDIR:, which when parsed will cause make to
change to a new object directory and reset .OBJDIR, and PWD in the
environment. This will allow some makefiles (mainly, src/tools)
to override the default objdir semantics in order to add custom logic.
 1.38  31-Oct-2001  tv Allow "-m" to be used in a .MAKEFLAGS: special target and get it to work.
(This splits out the "default system include paths" into its own Lst
variable, and uses it only if sysIncPath is empty. This allows sysIncPath
to be filled in by the Makefile itself.)
 1.37  16-Sep-2001  wiz Spell 'occurred' with two 'r's.
 1.36  03-Jul-2001  christos add .USEBEFORE Attribute
 1.35  09-Jun-2001  sjg Modify handling of command line variable assignments and their exporting
via MAKEFLAGS. Instead of appending them directly to .MAKEFLAGS, put
them in .MAKEOVERRIDES (and ensure they are quoted). This is now done
in Var_Set when it exports VAR_CMD's.
Use ExportMAKEFLAGS() to export MAKEFLAGS, using the combined content
of .MAKEFLAGS and .MAKEOVERRIDES (with duplicate supression).
If .MAKEFLAGS is assigned to in a Makefile, ExportMAKEFLAGS is called again.
This allows a line like:
.MAKEOVERRIDES=
to effectively stop the exporting of the command line vars in MAKEFLAGS.
 1.34  01-Jun-2001  sjg A number of semi-related changes.
1. make -dx turns on DEBUG_SHELL which causes sh -x to be used where
possible.
2. PrintOnError() is now called when make is stopping due to an error.
This routine reports the curdir and the value of any variables listed
in MAKE_PRINT_VAR_ON_ERROR.
3. Variables set via command line, are propagated to child-makes via
MAKEFLAGS. This behaviour appears to be necessary for POSIX (according
to the GNU folk anyway).
4. Do not reset MAKEFILE when reading ".depend" as this rather eliminates the
usefulness of ${MAKEFILE}.
5. Added ${.newline} as a simple means of being able to include \n in the
result of a :@ loop expansion.
6. Set ${MAKE_VERSION} if defined. Need to come up with a useful value.

Reviewed: christos
 1.33  14-Jan-2001  christos - make the second list of commands override the first like gmake does
- remember where the first command definition occured so that we can debug
things.
 1.32  10-Jan-2001  christos PR/11800: Chris Demetriou: Add -W flag to exit on parse warnings.
While I am here, s/make:.*"/%s:\1", progname/
 1.31  01-Jan-2001  sommerfeld Missing bit of last change
 1.30  30-Dec-2000  sommerfeld Add a token-passing scheme to allow a recursive make to successfully
use -j; all make's in a recursive build cooperate to limit the total
number of jobs, using a token-passing scheme.

The current token passing algorithm is similar to the one implemented
by gmake; there is a single pipe which is inherited through the entire
process hierarchy; tokens are obtained by reading a byte from the
"read end" of the pipe, and are returned by writing them to the "write
end". This exact algorithm is likely to change in the future.

Implementation details:
- Use the new trace facility to allow measurement of the
effectiveness of different token-passing schemes
- Get a token in MakeStartJobs(), return it in Make_Update()
- Eliminate Job_Full() and the jobFull global since they are
redundant with token system.
- Add an "internal" -J option (to pass the token pipe fd's down to
submakes) and a -T option for tracing.
- Change how compatMake is forced so that -j means something when
inherited by submakes.
- When waiting for a token, poll the token-passing pipe as well as
the output pipes of existing jobs.
 1.29  11-Jun-2000  mycroft Readd optimization last night. Problems earlier were partially due to the
arguments names on one function being swapped (by a previous author).

Do not do any duplicate suppression when a source list is created. Instead:
* OP_MADE protects against trying to make the source multiple times.
* A new OP_MARK flag is introduced to suppress duplicates while expanding
the .ALLSRC variable and .USE targets.
This turns the O(n^2) insertion into O(n) in most cases.

This is tested with a `make build' and some special test cases.
 1.28  10-Jun-2000  mycroft Back out last night's optimization for now.
 1.27  10-Jun-2000  mycroft Introduce an OP_MARK bit, and use it to suppress duplicates during .ALLSRC
and .USE expansion. Also, remove some more Lst_Member() checks that are now
redundant.
 1.26  01-Jun-2000  mycroft Create a `for' context, and substitute iteration variable from it using the
normal Var_Parse() path. This allows :R, etc. to work on iteration variables.
 1.25  04-May-2000  drochner branches: 1.25.2;
make cleaner C for more picky compilers
 1.24  20-Apr-2000  sjg Changes to make MAKEOBJDIRPREFIX useful.

Firstly, we ignore getenv("PWD") if MAKEOBJDIRPREFIX is set so that we always
get the same value for .CURDIR regardless of how make was invoked.

Second, when executing a command we check if it is ${.MAKE} or ${.MAKE:T}
without a preceeding chdir, if so we insert a chdir(${.CURDIR}) so that
the Makefile will be found by the child make. Note that this behaviour is
dissabled if MAKEOBJDIRPREFIX is not set or if NOCHECKMAKECHDIR is set.
See the comments in main.c for more detail.

With these two changes, one can successfully build usr/src using MAKEOBJDIRPREFIX
allowing the src to be mounted from a CD-ROM.
 1.23  16-Sep-1999  kim Make this compile on Solaris...
 1.22  15-Sep-1999  mycroft Replace the linear variable list in the GNode with a hash table. Gives a
small (~10% on my machine) speedup building libc.
Changes by: perry (+ bug fix and minor change by me)
Reviewed by: mycroft sommerfeld
 1.21  11-Nov-1998  christos This patch fixes the problem introduced in the previous commit where
parents would be get remade, even if children were not really updated
by the commands executed for them. It also makes all the children have
the real modification time set if possible, so it should fix some other
timing weirdnesses...

- collapse childMade and make fields into flags and convert them to bits
CHILDMADE and REMAKE
- introduce FORCE flag that gets set in all the parents of a child that
has no sources and does not exist.
- set oodate if the FORCE flag is set, and not if CHILDMADE
- centralize the RECHECK into Make_Recheck() and use this in make.c and compat.c
- use Make_TimeStamp for all child -> parent timestamp propagations
 1.20  18-Oct-1998  christos Only define const and volatile when __STDC__ is not defined (for BOOTSTRAP)
 1.19  21-May-1998  tv Make a MAKE_BOOTSTRAP case work on NetBSD, too, for cross-compile cases
 1.18  23-Sep-1997  fair fix some ints that should be "time_t" per PR#4139
 1.17  08-May-1997  gwr Add the new .NOPATH feature which can be used to disable .PATH search
for particular targets, i.e. .depend, objects, etc. (from Christos).
 1.16  02-May-1997  christos - Target searching addition:
Make used to only use the search path for nodes that were pure
sources (not targets of other sources). This has been corrected
and now gnu-autoconf generated Makefiles work in directories other
than the source one.

- Suffix transformation rescanning:
Suffix transformations (.c.o:; cc ...) were only recognized in
the past when both suffixes were members of the suffix list.
Thus a sequence like:
.z.b:
echo ${.TARGET}
.SUFFIXES: .z
would cause .z.b: to be inserted as a regular target (and the main
target in this case). Other make programs always add rules that
start with a period in the transformation list and never consider
them as targets. We cannot do that (consider .depend files) so we
resort to scanning the list of the current targets every time a
suffix gets added, and we mutate existing targets that are now
valid transformation rules into transformation rules. If the
transformed target was also the main target, we set the main target
to be the next target in the targets list.
 1.15  10-Mar-1997  christos Add a .MADE directive to indicated that the children of a target are
up-to-date, even when they are not. This is to simulate our current
make install behavior with proper dependencies.
 1.14  20-Feb-1997  christos Reported by cgd: fix .USE directive problems:
1. ${.*} variables did not get expanded in dependencies.
2. expanded ${.*} variables in .USE dependencies can cause tree
restructuring; handle it.
3. in compat mode, expand .USE before evaluating the list of targets,
instead of doing .USE expansions on demand, because they can cause
tree restructuring.
 1.13  31-Dec-1996  christos BSD did not traditionally have <sys/cdefs.h>; use BSD4_4 instead and include
<sys/param.h> to grab it.
 1.12  31-Dec-1996  christos define const and volatile for non __STDC__
 1.11  06-Nov-1996  christos - Merge in FreeBSD and Lite2 changes.
- Fix bug where a non-archive target with a .a suffix would always
be considered to be out of date, since it does not have a TOC.
 1.10  13-Aug-1996  christos Protect __P from being multiply defined (for systems that already define it)
 1.9  06-Mar-1996  christos branches: 1.9.4;
Fix PR/2118; apply sysincludes patch.
 1.8  04-Feb-1996  christos fix pr/1421 and pr/1997
 1.7  16-Dec-1995  christos - Added .WAIT to synchronize between sources like other pmake variants.
- Updated documentation to include .ORDER .PARALLEL .NO_PARALLEL .NONPARALLEL
 1.6  14-Jun-1995  christos - $NetBSD$ rcsids
- Fixed so that .[A-Z]* targets that do not match keywords are ignored as
Posix mandates
- Added .PHONY target keyword
 1.5  09-May-1994  glass bootstrap improvements
 1.4  17-Apr-1994  glass bootstrap changes
 1.3  05-Mar-1994  cgd fixes/improvements from Christos Zoulas <christos@deshaw.com>.
 1.2  01-Aug-1993  mycroft Add RCS identifiers.
 1.1  21-Mar-1993  cgd branches: 1.1.1;
Initial revision
 1.1.1.2  28-Dec-1996  tls Import 4.4BSD-Lite2 sources onto CSRG branch (already merged at head)
 1.1.1.1  21-Mar-1993  cgd initial import of 386bsd-0.1 sources
 1.9.4.1  26-Jan-1997  rat Update make(1) from trunk, by request from Christos Zoulas. Fixes many bugs.
 1.25.2.1  23-Jun-2000  minoura Sync w/ netbsd-1-5-base.
 1.64.2.2  05-Jun-2007  bouyer Apply patch (requested by tron in ticket #696):
usr.bin/make/compat.c patch
usr.bin/make/cond.c patch
usr.bin/make/dir.c patch
usr.bin/make/for.c patch
usr.bin/make/main.c patch
usr.bin/make/make.1 patch
usr.bin/make/make.c patch
usr.bin/make/make.h patch
usr.bin/make/nonints.h patch
usr.bin/make/parse.c patch
usr.bin/make/str.c patch
usr.bin/make/targ.c patch
usr.bin/make/util.c patch
usr.bin/make/var.c patch

Synchronize make(1) with HEAD branch to increase perfomance and
improve stability.
 1.64.2.1  20-Dec-2006  bouyer branches: 1.64.2.1.2;
Pull up following revision(s) (requested by dsl in ticket #285):
usr.bin/make/cond.c: revision 1.36
usr.bin/make/parse.c: revision 1.121-1.124
usr.bin/make/make.h: revision 1.65
usr.bin/make/targ.c: revision 1.46
usr.bin/make/main.c: revision 1.138-1.139
Simplify the code that processes .if lines.
It doesn't need a two-dimensional array to remember the states of .if lines.
It would be even simpler if we didn't try to detect .else and .elif lines
that follow .else lines.
Unfortunately this isn't the code that is stupendously slow...
Unknot this code slightly by avoiding 'break/return; else' and by putting
the short parts of some conditionals first.
First step towards speeding up the parsing of makefiles (esp. the unwanted
parts of .if clauses).
There should be no changes to the logic.
There really is no point calling estrdup() to get a copy of the makefile name
into every 'gnode' (aka target), instead just copy a pointer to the string
and avoid freeing the original name when we close the file.
I can't imagine any makefile set where this gives a larger footprint!
Add anewline to the end of the error message output when the debug log
file cannot be opened.
Put the big block of code that was common to ParseDoInclude() and
ParseTraditionalInclude() into a separate routine.
Fix the 'use after free' and 'free on item not malloced' that got added
to ParseTraditionalInclude() in Feb 2006 (rev 1.111).
Kill the 'PTR' struct and put both its members into IFile.
Remove the parameter from ParseEOF(), 1 of the 3 calls passed the wrong value!
Fortunately another test stopped anything nasty happening, we'll use that
test instead.
Revert part of a recent commit.
ParseEOF() shouldn't close the original file.
Some minor changes:
- Only test 'printVars' once.
Has side effect of not trying to find the 'main' target for '-V varname'.
- Only reap the single child process in Cmd_Exec(),
I think this is for 'xx != cmd' so probobably no other children should exist.
- Don't read and parse .depend if '-r' and '-V varname' are set.
I suspect that .depend shouldn't affect the output of any -V command...
 1.64.2.1.2.1  03-Sep-2007  wrstuden Sync w/ NetBSD-4-RC_1
 1.67.4.3  23-Mar-2008  matt sync with HEAD
 1.67.4.2  09-Jan-2008  matt sync with HEAD
 1.67.4.1  06-Nov-2007  matt sync with HEAD
 1.84.2.1  05-Mar-2011  bouyer Sync with HEAD
 1.87.2.2  22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.87.2.1  30-Oct-2012  yamt sync with head
 1.89.2.2  20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.89.2.1  23-Jun-2013  tls resync from head
 1.92.2.1  10-Aug-2014  tls Rebase.
 1.100.2.1  07-Jan-2017  pgoyette Sync with HEAD. (Note that most of these changes are simply $NetBSD$
tag issues.)
 1.102.6.1  07-May-2021  martin Pull up following revision(s) (requested by mrg in ticket #1678):

bin/pax/options.c: revision 1.119
usr.bin/config/scan.l: revision 1.32
usr.bin/make/main.c: revision 1.274
bin/pax/extern.h: revision 1.60
usr.bin/config/defs.h: revision 1.106
usr.bin/make/make.h: revision 1.105
sbin/newfs_udf/udf_create.c: revision 1.27 (patch)
sbin/newfs_udf/newfs_udf.h: revision 1.7 (patch)
usr.bin/config/main.c: revision 1.100
usr.sbin/installboot/Makefile: revisions 1.53,1.54 (patch)
usr.sbin/installboot/installboot.h: revision 1.41 (patch)
usr.sbin/installboot/machines.c: revisisons 1.41,1.42 (patch)

Avoid depending on common symbols.
 1.104.6.1  30-Apr-2021  martin Pull up following revision(s) (requested by mrg in ticket #1263):

bin/pax/options.c: revision 1.119
usr.bin/config/scan.l: revision 1.32
usr.bin/make/main.c: revision 1.274
bin/pax/extern.h: revision 1.60
usr.bin/config/defs.h: revision 1.106
usr.bin/make/make.h: revision 1.105
sbin/newfs_udf/udf_create.c: revision 1.27 (patch)
sbin/newfs_udf/newfs_udf.h: revision 1.7 (patch)
usr.bin/config/main.c: revision 1.100

Avoid depending on common symbols.
 1.104.4.1  08-Apr-2020  martin Merge changes from current as of 20200406

RSS XML Feed