Home | History | Annotate | Download | only in make
History log of /src/usr.bin/make/var.c
RevisionDateAuthorComments
 1.1172  16-Sep-2025  sjg Allow compiler to set size of hexdigits
 1.1171  29-Jun-2025  rillig make: fix parsing of modifier parts for :gmtime and :localtime

There's no practical use for escaping the ":" or "}" in these two
modifiers, but the same scheme may prove useful for the :M and :N
modifiers, which are currently parsed in a completely different manner,
thus adding needless complexity.

Parsing of the :M and :N modifiers is discussed in PR bin/46139.
 1.1170  28-Jun-2025  rillig make: consistently use double quotes in error messages

Previously, some error messages used single quotes, very few used
backticks or parentheses, most already used double quotes.

While here, reword a few error messages to be clearer or more consistent
with related error messages.
 1.1169  28-Jun-2025  rillig make: add POSIX $^ support

POSIX 2024 [1] introduced the $^ internal macro. Implement it as an
alias for $> (.ALLSRC), following the suggestion from defect 1520 [2].

[1]: https://pubs.opengroup.org/onlinepubs/9799919799/utilities/make.html
[2]: https://www.austingroupbugs.net/view.php?id=1520

Patch provided by Diogo Santos in PR bin/59018.
 1.1168  13-Jun-2025  rillig make: skip inter-process stack trace when MAKE_STACK_TRACE=no

Previously, setting MAKE_STACK_TRACE to any other value than "yes"
accidentally generated almost-complete inter-process stack traces.
 1.1167  13-Jun-2025  rillig make: add on-demand inter-process stack traces

When debugging a build that heavily uses nested calls to sub-makes, the
chain of calls is useful to quickly assess the situation. If explicitly
requested, include the actions from any parent processes in the stack
traces.

Reviewed by: sjg
 1.1166  12-Jun-2025  rillig make: use a common style for unexpected error messages

In enomem, report the actual error instead of a fixed ENOMEM.
 1.1165  11-Jun-2025  sjg make: check variable names for invalid characters

Variable names should not contain white-space.
As is; make quietly ignores gmake syntax like
$(addprefix -I, $(INCLUDE_DIR_LIST))
which means we are not doing what the makefile wants, and the
user has no clue.

CheckVarname is called only if there are no modifiers to be applied
as :L and others legitimately can handle white-space,
it will issue a warning if illegal characters are found.
Currently only white-space is checked.

The NetBSD build produces a few warnings - in one case a makefile
intended for gmake, and the other some dubious constructs by bsd.prog.mk
in support of ATF tests such as OBJS.${PROG} when PROG is actually a list.

Reviewed by: rillig
 1.1164  26-May-2025  rillig make: prepare for inter-process stack traces

To understand how chains of sub-makes interact and how they are called,
it's useful to have stack traces that span multiple processes. To
implement this, a stack trace needs not only be printed but also stored
as a string.
 1.1163  26-May-2025  rillig make: show contents of MAKEFLAGS in the stack trace

When the internal -J option refers to a closed file descriptor, it's
helpful to know what the MAKEFLAGS environment variable contains, in
addition to the executed command line. Same for parse errors in
variable assignments from the command line.
 1.1162  03-May-2025  rillig make: miscellaneous cleanups
 1.1161  30-Apr-2025  rillig make: fix order of error messages in the ":?" modifier

While here: In Var_Parse, EvalStack_Pop needs to be called before
freeing the short-lived variable, as its name is part of the evaluation
stack.
 1.1160  22-Apr-2025  rillig make: clean up

Replace 'unsigned int' with simply 'unsigned'.

In compat.c, skipping whitespace is not needed, as the loop above
already skips it.

In job.c, remove the unused header <sys/file.h>.

Inline the TMPPAT macro, as it is only needed in a single place.
 1.1159  04-Apr-2025  rillig make: fix error message for undefined variable in conditional

Previously, only ${UNDEF} had the proper error message, while
${UNDEF:M*} still produced the wrong "Malformed conditional".
 1.1158  30-Mar-2025  sjg make: POSIX mode, check for sysV style modifiers first

A POSIX compatible makefile should not be using any
of our native modifiers, so give preference to the sysV style modifier
and only check the others as a fallback.

Reviewed by: rillig
 1.1157  30-Mar-2025  rillig make: fix error message for invalid ":[...]" modifier

The unparsed value of the modifier argument can already be seen in the
stack trace, but the modifier argument may contain expressions and the
expanded value of these expressions was hidden up to now. See the EMPTY
test at the bottom of varmod-select-words.mk for details.
 1.1156  30-Mar-2025  rillig make: add details to error message for the "::=" modifier

The previous error message about a bad modifier ":" was not helpful, as
the strcspn call stopped immediately due to the modifier starting with
the separater character ":". The previous error message also didn't
reveal that the "parse error" was due to the expression name being
empty.
 1.1155  30-Mar-2025  rillig make: simplify error handling for unknown modifiers
 1.1154  30-Mar-2025  rillig make: let the ":t" modifiers fall back to the ":from=to" modifier

Suggested by https://bugs.freebsd.org/285726.

This means that the ":t" modifiers cannot have future extensions that
include a "=", as that may break existing code.
 1.1153  29-Mar-2025  rillig make: let unknown ":O" modifiers fall back to the ":from=to" modifier

Inspired by https://bugs.freebsd.org/285726, which concerns the ":t"
modifier instead.

This means that future extensions to the ":O" modifier must not contain
a "=" anywhere, otherwise they may break existing code.
 1.1152  29-Mar-2025  rillig make: in a stack trace with details and includes, list the current file
 1.1151  29-Mar-2025  rillig make: add ":" to error message about unknown modifier

In the manual page, the modifiers are listed with a preceding ":", so
use the same pattern in the error message. This removes an inconsistency
between the error messages "Unknown modifier" and "Bad modifier".
 1.1150  29-Mar-2025  rillig make: stop parsing after seeing an unknown modifier in an expression

Previously, after an expression such as ${VAR:Z::::}, make detected the
unknown modifier ":Z" and then continued parsing, which produced
unnecessary follow-up error messages. It was also necessary to
distinguish the error cases when logging the result of an applied
modifier in -dv mode.

Unify the error handling cases of a syntax error, an evaluation error
and an unknown modifier, to avoid the unnecessary follow-up error
messages.

The test in varmod-edge.mk now produces ":}" from the erroneous
expression, which may be misleading and thus will be looked at in a
follow-up commit.

The general idea of this patch was reviewed by sjg, I made a few
nonsubstantial changes after the review.
 1.1149  29-Mar-2025  rillig make: fix error message for unclosed expression

Even in an unclosed expression such as "${VAR:from=to", the modifier
":from=to" needs to be recognized as such, instead of giving an error
message about an "Unknown modifier ":from=to"".
 1.1148  29-Mar-2025  rillig make: add more details to error message about unfinished modifier

These details allow to quickly see the place where the syntax error is,
based on the surrounding lines from the stack trace.
 1.1147  29-Mar-2025  rillig make: add details about indirect modifiers to the stack traces

Previously, the error message "Unfinished modifier (',' missing)" from
moderrs.mk didn't provide enough context to understand where and why the
comma was missing.
 1.1146  29-Mar-2025  rillig make: in stack traces from target commands, add the command level

A target can contain several commands, and these commands are likely to
contain the same expressions. To distinguish them, add one more line to
the stack trace, to narrow down the source of the error.
 1.1145  22-Mar-2025  rillig make: ignore attempts to override a read-only global with the same value

Up to NetBSD 9, make passed around the internal variable .MAKE.LEVEL.ENV
via MAKEFLAGS. Allow these old versions of make to coexist with newer
versions of make. See unit-tests/varname-dot-make-level.mk for details.

Fixes PR pkg/59184.
 1.1144  11-Jan-2025  rillig make: replace "Malformed conditional" with "Variable is undefined"

Evaluating a conditional such as ".if ${UNDEF}" produced a "Malformed
conditional" error, even though the form of the conditional was correct,
it was the content of the variable that was wrong.

Replace this message with the more accurate "Variable is undefined",
revealing which of the possibly many variables is actually undefined.
 1.1143  11-Jan-2025  rillig make: omit redundant follow-up message "Malformed conditional"

If there is a parse error or an evaluation error during the conditional,
the conditional is already mentioned in the "while evaluating condition"
line, so there's no reason to repeat it once again.

Same for the "Bad condition" message from the '?:' modifier.
 1.1142  31-Dec-2024  rillig make: reduce pointer indirections when unexporting a variable
 1.1141  23-Nov-2024  rillig make: fix confusing error message when overriding a read-only variable
 1.1140  31-Aug-2024  rillig make: simplify printing of an evaluation stack element
 1.1139  29-Aug-2024  rillig make: reduce line length in error messages

The error messages for deeply nested parse errors were hard to decipher,
due to the large amount of text. Split these messages into individual
lines, just as in the backtrace for .include files and .for loops. This
unified backtrace makes the output more uniform.
 1.1138  27-Aug-2024  rillig make: treat recursive variables non-fatally

A recursive variable is no worse than an unknown modifier, so treat them
in the same way by continuing parsing until the end of the makefile.
 1.1137  25-Aug-2024  rillig make: add more context to error message about recursive variables
 1.1136  20-Jul-2024  rillig make: remove wrong error message about an undefined variable
 1.1135  09-Jul-2024  rillig make: error out on syntax errors in ':M' and ':N' modifiers

More than a year ago, the warning has been added. Now it has been
promoted to an error.
 1.1134  07-Jul-2024  rillig make: move initialization of variable scopes to targ.c

The variable scopes are freed by Targ_End, so initialize them there as
well. Separate printing statistics and freeing memory, which makes
Var_End unnecessary.
 1.1133  05-Jul-2024  rillig make: in error messages, distinguish parsing from evaluating
 1.1132  05-Jul-2024  rillig make: in error messages for anonymous variables, log the value
 1.1131  05-Jul-2024  rillig make: error out on unclosed expressions during parse time

In exchange, this adds location information.

For unnamed expressions, the value is no longer printed. This will be
added back in a follow-up commit.
 1.1130  05-Jul-2024  rillig make: reduce lint-specific comments about ARGSUSED
 1.1129  04-Jul-2024  rillig make: error out on the "Bad modifier" error message

Previously, the "Bad modifier" error message did not affect make's exit
status. As a side effect, this kind of error now gets more context
information.
 1.1128  04-Jul-2024  rillig make: add more context information to error messages

In case of a parse error or evaluation error, print the variable value
in addition to the variable name, to see the effects of previous
expression modifiers.

In nested make calls, print the current directory at the bottom of a
stack trace, as that information is otherwise hard to get in a parallel
build spanning multiple directories.
 1.1127  02-Jul-2024  rillig make: trim trailing whitespace
 1.1126  01-Jul-2024  sjg make: add :tc to capitalize first letter of each word

This is very hard to do without :tc

Reviewed by: rillig
 1.1125  30-Jun-2024  rillig make: error out on some more syntax errors

Previously, these errors only produced a message on stderr. They only
affected make's exit status when they were evaluated at parse time, but
not when evaluating the commands for a specific target right before
executing them.

The affected syntax errors are:
* invalid regular expressions in the ':C' modifier
* out-of-range references to regex groups in the ':C' modifier
* unfinished modifiers
 1.1124  30-Jun-2024  rillig make: error out on syntax error in conditions in ':?then:else' modifier

The 'Error' function only reports errors but does not affect the exit
status, the 'Parse_Error' function does, while providing more details to
find the cause of the syntax error.
 1.1123  30-Jun-2024  rillig make: sync error handling between '!=' assignment and '::!=' modifier
 1.1122  30-Jun-2024  rillig make: add more context to "returned non-zero status" message

Previously, this message was an "error" but had no influence on the exit
status, so make it a warning instead. In the seldom used -W mode that
treats warnings as errors, this change influences the exit status.
 1.1121  15-Jun-2024  rillig branches: 1.1121.2;
make: error out if an internal read-only variable is overwritten

The affected variables are:
* .newline
* .MAKE.OS
* .MAKE.JOBS.C
* .MAKE.LEVEL.ENV
* .MAKE.PID
* .MAKE.PPID
* .MAKE.UID
* .MAKE.GID

This change does not affect the .SHELL and .SUFFIXES variables, as well
as variables that are manually turned read-only by the special .READONLY
target.
 1.1120  15-Jun-2024  rillig make: clean up collection of context information for error messages
 1.1119  02-Jun-2024  rillig make: sync VarEvalMode constant names with their debug log names
 1.1118  02-Jun-2024  rillig make: fix out-of-bounds read when parsing indirect modifiers
 1.1117  01-Jun-2024  sjg make: add .export-all

An explicit syntax for exporting all global variables is much safer
than allowing .export with no argument to do the same.

Add .export-all and have .export with no argument throw a warning saying
to use .export-all

Reviewed by: rillig
 1.1116  01-Jun-2024  rillig make: remove unused VARE_EVAL_KEEP_DOLLAR
 1.1115  31-May-2024  rillig make: merge functions that parse modifier parts

This commit subtly changes the behavior for the :gmtime and :localtime
modifiers, but only in irrelevant cases since the time value must be an
integer, and neither '\' nor ':' nor '$' may occur in these values.
 1.1114  31-May-2024  rillig make: clean up API for iterating over hash tables
 1.1113  30-May-2024  rillig make: don't log anything when freeing memory
 1.1112  25-May-2024  rillig make: fix some more memory leaks
 1.1111  25-May-2024  rillig make: fix a few more memory leaks
 1.1110  24-May-2024  rillig make: in -DCLEANUP mode, free variables and their values

The variables in the 3 scopes must be freed before the scopes themselves
are freed by Targ_End.

The test opt-m-include-dir creates a directory of the form '*.tmp', thus
it must be removed before attempting to only remove regular files of
this name.

POSIX requires setenv to copy the passed name and value, so there is no
need to keep that memory allocated any longer.
 1.1109  07-May-2024  sjg make: all command line overrides go in .MAKEOVERRIDES

Not all variables that start with '.' are internals,
and unless they are explicitly flagged as internal should go
into .MAKEOVERRIDES

Update varname-dot-makeoverrides to check this.

Also avoid using SCOPE_CMDLINE when ReadOnly will do.
 1.1108  28-Apr-2024  rillig make: don't reallocate memory after evaluating an expression

When an expression is evaluated, the resulting text is short-lived in
almost all cases. In particular, the compaction neither affects the
target names nor the global variable values, which are the prime
candidates for permanent memory usage.
 1.1107  27-Apr-2024  rillig make: fix comment about forcing a use-after-free

The previous expression didn't cause any bug, as the modifier
':@VAR@loop@' changed the value of the expression, thus making the
expression independent from the variable value.

Instead, the variable needs to be deleted from within an indirect
modifier, and that modifier needs to evaluate to an empty string, thus
doing nothing and preserving the original expression value.
 1.1106  27-Apr-2024  rillig make: clean up, test .NOPATH

Trim down the comments in the archive module, as they mainly repeated
the code. Trim down the binary code size in the archive module, as it
is rarely used.

In Var_Parse, delay two variable assignments until they are actually
needed.
 1.1105  23-Apr-2024  rillig make: clean up comments, code and tests
 1.1104  21-Apr-2024  rillig make: trim down code for parsing the :gmtime and :localtime modifiers

The :gmtime and :localtime modifiers are not used often and thus are not
time-critical. Exchange the custom code that parses an integer from a
substring for an additional memory allocation.

Thanks sjg@ for suggesting to avoid the custom parsing code.
 1.1103  21-Apr-2024  rillig make: fix out-of-bounds read when evaluating :gmtime and :localtime

The function TryParseTime takes a pointer to a string, but the LazyBuf
returns a Substring, which is not guaranteed to be null-terminated or
delimited. In TryParseTime, calling strtoul on the Substring read past
the end of the substring.

Noticed in the NetBSD build in libntp, where the :gmtime modifier is
used in two places with the same timestamp value, of which the first was
evaluated correctly and the second wasn't.

The bug was introduced in var.c 1.1050 from 2023-05-09, when the
argument of the :gmtime and :localtime modifiers was allowed to be an
expression instead of an integer constant.
 1.1102  20-Apr-2024  rillig make: provide more context information for parse/evaluate errors
 1.1101  01-Mar-2024  rillig make: fix type mismatch in lint's strict bool mode (since today)
 1.1100  01-Mar-2024  sjg make: export target scope values

Pass target scope to Var_ReexportVars so that a target process
will see the correct values in its env.
We must then mark any Global scope variable as unexported
so targets without local value get the Global one.
 1.1099  07-Feb-2024  rillig make: remove unneeded conditional-compilation toggles

The toggles INCLUDES, LIBRARIES, POSIX, SYSVINCLUDE, SYSVVARSUB,
GMAKEEXPORT and SUNSHCMD are no longer needed, they were unconditionally
set.

The toggle NO_REGEX was configurable from the command line, but
disabling it would result in various error messages about the unknown
':C' modifier.

OK sjg@.
 1.1098  04-Feb-2024  rillig make: do not evaluate indirect modifiers in parse-only mode

Discovered by sjg.
 1.1097  04-Feb-2024  rillig make: in parse-only mode, don't evaluate modifiers

Previously, the ':S', ':ts', ':tA' and ':from=to' modifiers were
evaluated in parse-only mode, unnecessarily. This is only noticeable
when an indirect modifier is evaluated in parse-only mode, which is
another bug that will be fixed in a follow-up commit.
 1.1096  03-Feb-2024  sjg make: state of save_dollars affects what is a parse error

When save_dollars is false it is not a parse error to
encounter $$ rather than \$.
 1.1095  21-Jan-2024  rillig make: clean up redundant 'const' from automatic variables

No binary change.
 1.1094  07-Jan-2024  rillig make: clean up comments, constify shell name
 1.1093  05-Jan-2024  rillig make: miscellaneous cleanups
 1.1092  05-Jan-2024  rillig make: clean up string functions

No binary change.
 1.1091  05-Jan-2024  rillig make: inline Substring_Sub

Remove redundant assertions.
 1.1090  29-Dec-2023  rillig make: simplify debug message for the ':@var@...@' modifier

The previous variant was hard to understand.
 1.1089  29-Dec-2023  rillig make: clean up variable handling

All variables from the command line scope have the fromCmd flag set, so
there is no need to check for it.

Inline redundant local variables.

Variables from a scope cannot be short-lived, so there is no need to
call VarFreeShortLived.

No functional change.
 1.1088  29-Dec-2023  rillig make: clean up comments

No binary change, except for line numbers in assertions.
 1.1087  29-Dec-2023  rillig make: simplify memory allocation for string buffers

In edge cases and short-lived buffers, the initial buffer size is
irrelevant, so use the default.

No functional change.
 1.1086  20-Dec-2023  rillig make: use consistent debug messages style when ignoring variables

When a variable is not modified or not deleted, clearly say so and state
the reason. Use the same style of debug messages everywhere, putting
the word 'ignoring' at the front. Previously, that word sticked out to
the right, but only in some cases.
 1.1085  20-Dec-2023  rillig make: fix confusing debug logging when deleting a variable

Be clear when deleting a variable is ignored and state the reason,
instead of only providing a keyword as a hint for insiders.
 1.1084  19-Dec-2023  rillig make: clean up comments

No binary change, except for line numbers in assertions.
 1.1083  17-Dec-2023  rillig make: clean up names of local variables

No binary change.
 1.1082  10-Dec-2023  rillig make: explain in the debug log why a variable assignment is ignored
 1.1081  10-Dec-2023  rillig make: remove redundant text from diagnostic about bad '?:' modifier
 1.1080  10-Dec-2023  rillig make: clean up the check for command line variables

It looked suspicious that to check whether a variable was set via the
command line, the variable value would be needed. Moving the debug
diagnostic to the calling function resolved this smell.

A variable from the command line scope is never short-lived, so there's
no need to clean up after accessing the variable.

No functional change.
 1.1079  10-Dec-2023  rillig make: clean up comments and local identifiers

No binary change, except for line numbers in assertions.
 1.1078  10-Dec-2023  rillig make: document how to trigger a use-after-free bug

Since 2017-02-01.
 1.1077  10-Dec-2023  rillig make: fix crash in ':C' modifier on an empty expression

Since var.c 1.982 from 2021-12-13.
 1.1076  09-Dec-2023  sjg make: Var_SetWithFlags set fromCmd for SCOPE_CMDLINE

Set fromCmd true for SCOPE_CMDLINE
regardless of the variable name.

Reviewed by: rillig
 1.1075  19-Nov-2023  rillig make: replace 'variable expression' with 'expression' in comments

No binary change.
 1.1074  19-Nov-2023  rillig make: replace 'variable expression' with 'expression' in diagnostics
 1.1073  19-Nov-2023  rillig tests/make: cover all code paths for the ':mtime' modifier
 1.1072  19-Nov-2023  rillig make: produce more accurate error message for invalid ':mtime' argument
 1.1071  19-Nov-2023  rillig make: extract detection of ':from=to' to a separate function

No functional change.
 1.1070  19-Nov-2023  rillig tests/make: test and explain exporting of variables
 1.1069  18-Nov-2023  rillig make: clean up the modifier ':[...]'

No functional change.
 1.1068  02-Nov-2023  rillig make: when comparing substrings, don't read beyond the substring's end

Right now, Substring_Words terminates each word with a '\0', but that's
an implementation detail that is not required by the interface, so don't
rely on it.
 1.1067  02-Nov-2023  rillig make: miscellaneous cleanups

No functional change.
 1.1066  02-Nov-2023  rillig make: clean up comments

No functional change.
 1.1065  02-Nov-2023  rillig make: sync comments with the extracted ParseModifier_Match

No functional change.
 1.1064  19-Aug-2023  rillig make: only work around wrong strftime if actually necessary

The workaround is only needed if the time format contains '%s', in all
other cases there is no need to preserve, set and restore the TZ
environment variable. Suggested by sjg@.

Only check for 's' in the format string, not for '%s', to allow for
optional modifiers of the conversion specifier.
 1.1063  19-Aug-2023  rillig make: clean up comments regarding gmtime/strftime

These two functions are not supposed to be used together, thus it is not
a bug if combining them produces unexpected results.
 1.1062  19-Aug-2023  rillig make: work around bugs in gmtime on several platforms
 1.1061  17-Aug-2023  rillig make: be strict when parsing the argument of the ':mtime' modifier
 1.1060  17-Aug-2023  rillig make: clean up variable names and indentation for the ':mtime' modifier

No functional change.
 1.1059  23-Jun-2023  rillig make: clean up variable and function names

No functional change.
 1.1058  23-Jun-2023  rillig make: warn about malformed patterns in ':M', ':N' and '.if make(...)'

These patterns shouldn't occur in practice, as their results are tricky
to predict. Generate a warning for now, and maybe an error later.

Reviewed by sjg@.
 1.1057  22-Jun-2023  rillig make: merge common code for handling the ':M' and ':N' modifiers

No functional change.
 1.1056  16-Jun-2023  sjg make: do not allow delete of readOnly variable.

Sometimes we mark a variable readOnly to guard against .undef
 1.1055  01-Jun-2023  rillig make: shorten function names, clean up comments

No functional change.
 1.1054  10-May-2023  sjg Make :mtime operate on each word in variable value.

Reviewed by: rillig
 1.1053  09-May-2023  rillig make: improve error message for failed stat in ':mtime'
 1.1052  09-May-2023  sjg make: :mtime=error throw error on stat(2) failure

Sometimes we want fatal error if stat fails on the
presumed pathname.
 1.1051  09-May-2023  sjg make: add :mtime to provide mtime of file

The value of the variable is passed to stat(2)
and st_mtime is new value.
An optional arg can be used if stat(2) fails, otherwise
the current time is used.

See varmod-mtime.mk for usage examples.
 1.1050  09-May-2023  rillig make: allow ':gmtime' and ':localtime' with dynamic argument

This allows ${%Y:L:gmtime=${mtime}} instead of the indirect
${%Y:L:${:Ugmtime=${mtime}}}.

The direct form also prevents any ':' from the nested expression to be
interpreted as a separator, which doesn't matter for the ':gmtime' and
':localtime' modifiers but will prove useful for other modifiers that
follow the same pattern.
 1.1049  28-Mar-2023  rillig make: declare all common symbols in headers, unexport others

No functional change.
 1.1048  09-Mar-2023  rillig make: fix memory leak when parsing ':from=to' modifiers

Since var.c 1.925 from 2021-04-12.
 1.1047  18-Feb-2023  rillig make: fix parsing of unevaluated subexpressions with unbalanced '{}'

Since var.c 1.323 from 2020-07-26, modifiers containing unbalanced
braces or parentheses were parsed differently, depending on whether they
were relevant or not.

For example, the expression '${VAR:...}' is enclosed with braces. When
this expression has a modifier ':S,},}},g' that would double each '}' in
that expression, the parser got confused:

If the expression was relevant, the modifier was parsed as usual, taking
into account that the 3 '}' in the modifier are ordinary characters.

If the expression was irrelevant, the parser only counted the '{' and
the '}', without taking into account that a '}' might be escaped by a
'\' or be an ordinary character. Parsing therefore stopped at the first
'}', assuming it would finish the expression '${VAR:S,}'.

This parsing mode of only counting balanced '{' and '}' makes sense for
the modifier ':@var@...@', which expands each word of the expression
using the template from the '...'. These templates tend to be simple
enough that counting the '{' and '}' suffices.
 1.1046  15-Feb-2023  rillig make: inline macros for variable names

The variable name '.MAKEOVERRIDES' was already used in the non-macro
form.

No binary change.
 1.1045  15-Feb-2023  rillig make: inline macro for variable name ".MAKE.EXPORTED"

The variable name is distinctive enough to be searched directly in the
code instead of having a named constant for it.

No binary change.
 1.1044  14-Feb-2023  rillig make: remove redundant type VarParseResult

No functional change.
 1.1043  14-Feb-2023  rillig make: clean up calls to Var_Subst

None of the calls to Var_Subst used the return value, and the return
value was always VPR_OK.

No functional change.
 1.1042  14-Feb-2023  rillig make: reduce complexity of evaluating expressions

No functional change.
 1.1041  13-Feb-2023  rillig make: extract the decision of using a specific log format

No functional change.
 1.1040  09-Feb-2023  sjg make: ApplyModifier_Remember always use SCOPE_GLOBAL

Conditionals evaluate vars with SCOPE_CMDLINE, if :_
is used within a conditional expression and is thus
set in SCOPE_CMDLINE, it breaks any use of :_ in SCOPE_GLOBAL.

The simplest solution is for :_ to always use SCOPE_GLOBAL

Reviewed by: rillig
 1.1039  26-Jan-2023  sjg make: some variables should be read-only

Make variables like .newline and .MAKE.{GID,PID,PPID,UID} read-only.

Reviewed by: rillig
 1.1038  24-Jan-2023  sjg make: log adjustments of var.readOnly
 1.1037  23-Jan-2023  sjg make: .[NO]READONLY for control of read-only variables

Reviewed by: rillig
 1.1036  05-Dec-2022  rillig make: inline LazyBuf_AddBytesBetween

No binary change.
 1.1035  01-Oct-2022  rillig make: constify, fix comment indentation

No functional change.
 1.1034  01-Oct-2022  rillig make: merge duplicate code

No functional change.
 1.1033  27-Sep-2022  rillig make: set WARNS to 6, from the default 5

No binary change on x86_64.
 1.1032  24-Aug-2022  rillig make: prevent future out-of-bounds errors when parsing expressions

A modifier in an expression ends not only at the next ':' or at the
closing '}' or ')', but also at the end of the string.

Previously, testing for the end of the string had been done separately,
which was error-prone since 2006-05-11, when indirect modifiers were
introduced. Since then, it was possible that the string terminator '\0'
was accidentally skipped in cases where the loop condition only tested
for the ending character. When parsing indirect modifiers, the ending
character is indeed '\0', but when parsing direct modifiers, it is '}'
or ')'.

A welcome side effect is that in the case of unclosed expressions such
as '${VAR:Modifier', the amount of error messages is reduced from 2 or 3
to only 1. The removed error messages were wrong and thus confusing
anyway.
 1.1031  24-Aug-2022  rillig make: fix out-of-bounds read when parsing the ':M' modifier

Since at least 2009-01-17, probably already since 2006-02-18, when
modifiers were allowed to be nested expressions.
 1.1030  24-Aug-2022  rillig make: fix out-of-bounds read when parsing the ':D' modifier

Since 2000-04-29, when the ':D' and ':U' modifiers were added.
 1.1029  23-Aug-2022  rillig make: revert parsing of modifier parts (since 2022-08-08)

The modifier ':@var@body@' parses the body in parse-only mode and later
uses Var_Subst on it, in which each literal '$' must be written as '$$'.

Trying to parse the loop body using Var_Parse treated the text
'$${var:-0}' as a single '$' followed by the expression '${var:-0}',
wrongly complaining about the 'Unknown modifier "-0"'.

Found by sjg.
 1.1028  08-Aug-2022  rillig make: fix parsing of modifiers containing unbalanced subexpressions
 1.1027  05-Aug-2022  rillig make: when parsing ':D' or ':U', only copy text if necessary

When parsing the expression ${:D any ${uninterpreted} text}, parsing of
the ':D' modifier only needs to skip to the next delimiter, there is no
need to keep record of the text that has been skipped. In this
particular example, there had been an unnecessary memory allocation
since the nested expression was not copied to the result buffer. Due to
that, the resulting buffer contained " any text", which was not a
substring of the text passed to the parser, thereby forcing the memory
allocation.

No functional change.
 1.1026  05-Aug-2022  rillig make: extract parsing of ':D' and ':U' modifiers into separate function

No functional change.
 1.1025  14-Jun-2022  rillig make: document parsing of short variable names, such as $i

No binary change.
 1.1024  14-Jun-2022  rillig make: reduce indentation in ParseVarnameShort

No functional change.
 1.1023  14-Jun-2022  rillig make: simplify return type of IsShortVarnameValid

No functional change.
 1.1022  12-Jun-2022  rillig make: reorganize Parse_Error

Determining the location where the error occurred is now done by
ParseVErrorInternal. This frees the remaining code from keeping the
filename and the line number together. It also makes Parse_Error short
enough that it might be worth providing a separate function for each of
the 3 log levels.

No functional change.
 1.1021  14-May-2022  rillig make: clean up comments in Var_Parse
 1.1020  09-May-2022  rillig make: remove obsolete comment

All instances of VarFind that could be replaced with VarFindSubstring
have been replaced.
 1.1019  27-Mar-2022  rillig make: fix spacing, and a typo in a test
 1.1018  26-Mar-2022  rillig make: avoid trailing whitespace in debug log for variables

Since trailing whitespace is invisible, describe the variable value in
words to make it visible.
 1.1017  26-Mar-2022  rillig make: add space after colon in debug logging for variables

These log messages are intended for human interpretation, so don't make
them unnecessarily hard to read.
 1.1016  26-Mar-2022  rillig make: prefer 'long long' over 'long' on 32-bit C99 platforms

When sorting the words of an expression numerically using the modifier
':On' (added on 2021-07-30), use 64-bit numbers even on 32-bit
platforms. A typical use case is comparing file sizes.

When tracing the execution of jobs, fix an integer overflow after 2038.
32-bit platforms that use a pre-C99 compiler still have this problem.

No change to the test suite since most tests simply skip any potential
differences between 32-bit platforms and 64-bit platforms (see
varmod-order-numeric.mk) or already account for both variants (see
varmod-localtime.mk).
 1.1015  26-Mar-2022  rillig make: clean up comments
 1.1014  26-Mar-2022  rillig make: fix crash on .undef of an environment variable (since 2020-10-06)
 1.1013  03-Mar-2022  rillig make: simplify ParseModifier_Match

No functional change.
 1.1012  11-Feb-2022  rillig make: remove comment about environment variable with empty name

At least on NetBSD, make cannot be fooled with an environment variable
having an empty name. When running '/usr/bin/env =undefined make', the
argument is parsed as a variable assignment, but putenv(3) refuses to
process an empty variable name. Calling execve(2) directly got a step
further, the kernel didn't filter '=undefined' from the environment
variables. But getenv(3) always returns NULL when querying the
environment variable with the empty name.

On other operating systems, things may be different. Trying to set an
environment variable with an empty name may cause errors in env(1),
putenv(3), execve(2), getenv(3) or other places, so don't add an
automatic test for now.
 1.1011  09-Feb-2022  rillig make: simplify control flow in ModifyWord_SysVSubst

No functional change.
 1.1010  09-Feb-2022  rillig make: clean up variable names

No binary change.
 1.1009  04-Feb-2022  rillig make: use fixed type for comparing numbers using the modifier ':On'

When the modifier ':On' was added on 2021-07-30, there were concerns
that pre-C99 environments would not have the type 'long long', therefore
the type was made configurable, but parsing such numbers was hard-coded
to using strtoll.

To improve compatibility with C90 environments, use 'long' and 'strtol'
in these environments. In C99 environments, use 'long long' and
'strtoll', to account for larger file sizes.

If the flexibility of choosing yet another type for these numbers should
ever arise, it can still be implemented. Until then, reduce the number
of possible build configurations.
 1.1008  29-Jan-2022  rillig make: for recursive variables in commands, print location

Print the approximate location based on the last command that has been
defined for the target. It would be possible to get more detailed
location information by counting the number of commands of the target,
but that would get messy due to .USEBEFORE, .USE and .DEFAULT, and
still, this is an edge case, so don't waste too much code for it now.
Having this hint about the location is more helpful than just a plain
"Variable X is recursive" without any further details.
 1.1007  29-Jan-2022  rillig make: inline functions that have a single call site

No functional change.
 1.1006  27-Jan-2022  sjg Allow local variable assignments in dependency lines

The variable is set in the context of the target.
This syntax has been supported by gmake for ~ever.
If necessary a makefile can set .MAKE.TARGET_LOCAL_VARIABLES=false
to disable this.

Expose GetBooleanExpr so parse.c can use it.
 1.1005  24-Jan-2022  rillig tests/make: demonstrate that the 'static' in Var_Parse has an effect
 1.1004  24-Jan-2022  rillig make: remove unsatisfiable condition in VarSubstExpr

At that point, emode can have only 2 values, none of which is
VARE_UNDEFERR.

No binary change since GCC had figured this out all the time, it just
didn't reveal this little secret.
 1.1003  24-Jan-2022  rillig make: when expanding nested variables, check simple things first

No functional change, just a little performance improvement for
expressions that are not evaluated anyway but only parsed.
 1.1002  15-Jan-2022  rillig make: remove unnecessary functions for expanding variable names

In meta mode, the affected variable patterns do not contain a '$'.

Outside of meta mode, Global_SetExpand was only called a single time, so
inline that call.

No functional change.
 1.1001  15-Jan-2022  rillig make: merge duplicate code for expanding variable expressions

No functional change.
 1.1000  09-Jan-2022  rillig make: in Cmd_Exec, return error message instead of format string

This change leaves only literal format strings in parse.c. It allows
for more detailed error messages than the current "non-zero status" or
"exited on a signal".

No functional change.
 1.999  09-Jan-2022  rillig make: shorten ApplyModifier_Assign

No functional change.
 1.998  08-Jan-2022  rillig make: inline Buf_Clear

No functional change.
 1.997  08-Jan-2022  rillig make: rename is_shell_metachar, fix character conversion warning

No binary change.
 1.996  07-Jan-2022  rillig make: clean up comments, variable names, function names

The comment in ApplyDependencySourceOther repeated the code, its second
half didn't match any current code.

The comment above ParseDependencySourcesEmpty repeated the code.

No binary change, except for assertion line numbers.
 1.995  07-Jan-2022  rillig make: merge calls to ApplyModifier_Time

This enables GCC 11 to inline ApplyModifier_Time, like all the other
modifiers. Similar pattern as for ':M' and ':N', as well as for ':D'
and ':U'.

No functional change.
 1.994  07-Jan-2022  rillig make: merge duplicate code for modifiers 'gmtime' and 'localtime'

No functional change.
 1.993  07-Jan-2022  rillig make: clean up variable modifiers 'gmtime' and 'localtime'

Use consistent wording (zulu -> gmt), make VarStrftime parameter order
consistent with strftime, rename confusing 'time_t utc' to 't',
eliminate common subexpression in error message.

No functional change.
 1.992  30-Dec-2021  rillig make: condense code in ApplyModifier_Assign

List the assignment operators in the same order as in the manual page.

No functional change.
 1.991  29-Dec-2021  rillig make: merge duplicate types CondEvalResult and CondResult

No binary change.
 1.990  29-Dec-2021  rillig make: remove redundant parameter for evaluating conditions

No functional change.
 1.989  15-Dec-2021  rillig make: amend leftover cleanups from the previous commits

No functional change.
 1.988  15-Dec-2021  rillig make: format comments according to /usr/share/misc/style

Assisted by indent(1), with manual corrections due to its many remaining
bugs.

No functional change.
 1.987  15-Dec-2021  rillig make: remove space after ':' in bit-field declarations

As seen in /usr/share/misc/style.
 1.986  13-Dec-2021  rillig make: fix memory leak when parsing a variable name

To trigger the memory leak, the expanded variable name must not be a
prefix of the textual variable name.

textual name expanded name
ok: UNDEF.${undef} UNDEF.
leak: UNDEF.${undef}. UNDEF..
 1.985  13-Dec-2021  rillig make: reduce memory allocation in modifier ':?'

The memory allocation was in LazyBuf_DoneGet. The substring for the
untaken branch had been allocated even though it was not necessary.

No functional change.
 1.984  13-Dec-2021  rillig make: reduce memory allocations in modifier ':C'

The memory allocation was in the call to LazyBuf_DoneGet.

No functional change.
 1.983  13-Dec-2021  rillig make: condense RegexReplace for the modifier ':C'

No functional change.
 1.982  13-Dec-2021  rillig make: in the modifier ':C', don't call strlen unnecessarily

No functional change.
 1.981  13-Dec-2021  rillig make: extract RegexReplaceBackref from RegexReplace

No functional change.
 1.980  13-Dec-2021  rillig make: merge duplicate code for memory handling in Var_Parse

No functional change.
 1.979  13-Dec-2021  rillig make: fix remaining memory leak for .SUFFIXES (since yesterday)
 1.978  13-Dec-2021  rillig make: distinguish between short-lived and environment variables

No functional change.
 1.977  13-Dec-2021  rillig make: rename VarFreeEnv to VarFreeShortLived

The newly added "variable" .SUFFIXES is short-lived as well, which makes
it necessary to distinguish between environment variables and
short-lived variables.

No binary change.
 1.976  13-Dec-2021  rillig make: clean up code for freeing short-lived variables

Since yesterday's addition of the short-lived "variable" named
.SUFFIXES, not only environment variables are short-lived. Clean up the
code to prepare for fixing the remaining memory leaks.

No functional change.
 1.975  13-Dec-2021  rillig make: fix memory leak when evaluating ${.SUFFIXES} (since yesterday)

perl -e '
printf(".SUFFIXES: %s\n", "x" x 224);
printf("_:=\${.SUFFIXES}\n" x 100_000);
printf("all:\n");
' \
| MALLOC_CONF=stats_print:true \
./make -r -f - 2>&1 \
| awk '/bins:/,/^ *256/ { print $1 "\t" $4}'

Roughly 100000 less allocations for bin size 16, for the variable name.
 1.974  13-Dec-2021  rillig make: fix memory leak when evaluating ${.SUFFIX} (since yesterday)
 1.973  12-Dec-2021  sjg Add .SUFFIXES as read-only variable.

References to ${.SUFFIXES} are handled dynamically in
ParseVarnameLong by calling Suff_NamesStr.

The variable cannot be set normally.

Reviewed by: rillig
 1.972  12-Dec-2021  sjg ParseVarnameLong: minimize calls to LazyBuf_Get

Reviewed by: rillig
 1.971  11-Dec-2021  rillig make: inline ParseEmptyArg into CondParser_FuncCallEmpty

No functional change.
 1.970  09-Dec-2021  rillig make: in parse errors, mark whitespace more clearly

This prevents any trailing whitespace from going unnoticed. It also
marks leading whitespace more clearly, as in the examples with the time
value " 1".
 1.969  09-Dec-2021  rillig make: remove period from end of error messages and warnings

The majority of the existing error messages and warnings does not
include a period at the end. Follow this style consistently.
 1.968  07-Dec-2021  rillig make: merge branches for modifiers ':D' and ':U'

These branches have the exact same code, without involving any macros,
nevertheless GCC 10 doesn't merge them. Further, GCC thinks that the
function ApplyModifier_Defined were called from two different places,
thus creating an actual function definition. Merging the case labels
inlines the function, saving a few instructions and a few hundred bytes
in the binary.

No functional change.
 1.967  07-Dec-2021  rillig make: extract common code into Expr_Words

No functional change.
 1.966  06-Dec-2021  rillig make: move low-level implementation details out of Var_Parse

No functional change.
 1.965  06-Dec-2021  rillig make: extract common code into Expr_Str

This change turns Expr into a more abstract data type.

No functional change.
 1.964  05-Dec-2021  rillig make: fix types for modifier ':O' (since today)

In var.c 1.962 from 2021-12-05, the type of the array to be sorted
changed from 'const char *' to 'Substring'. The code still worked since
the first member of Substring is 'const char *start', but that was pure
luck.

Fix the type assumptions in the comparison functions.

Add an assertion that the words must be null-terminated since the
comparison functions currently assume this.

This kind of inconsistency may be possible to detect using static
analysis, but NetBSD's lint is currently too limited for that. To
detect this, it would have to analyze each possible comparison function
and remember the assumed type. This type could then be compared to the
argument type of a function call to qsort or bsearch.

No functional change.
 1.963  05-Dec-2021  rillig make: fix use-after-free in modifier ':@'

Without memory allocator debugging, the newly added test doesn't show
any obvious failure.

With memory allocator debugging enabled, all make versions since
2016.02.27.16.20.06 crash with a segmentation fault.
 1.962  05-Dec-2021  rillig make: save a memory allocation in each modifier ':O' and ':u'

No functional change.
 1.961  05-Dec-2021  rillig make: save a memory allocation in the modifier ':[...]'

No functional change.
 1.960  05-Dec-2021  rillig make: save a memory allocation in the modifier ':[...]'

No functional change.
 1.959  05-Dec-2021  rillig make: in UnexportVars, replace Str_Words with Substring_Words

This skips an allocation and a bit of memory copying.

No functional change.
 1.958  03-Dec-2021  rillig make: condense ParseModifierPartSubst

No functional change.
 1.957  03-Dec-2021  rillig make: extract ParseModifierPartExpr from ParseModifierPartSubst

No functional change.
 1.956  03-Dec-2021  rillig make: extract ParseModifierPartDollar from ParseModifierPartSubst

No functional change.
 1.955  28-Nov-2021  rillig make: fix type of variable in ApplyModifier_IfElse

No functional change.
 1.954  20-Nov-2021  rillig make: use correct plural form in debug message
 1.953  17-Oct-2021  rillig make: fix order of parameters in function comments

No functional change.
 1.952  23-Sep-2021  rillig make: fix memory leak in error case of the ':?' modifier
 1.951  21-Sep-2021  rillig make: reduce relocations, thereby reducing .text size

No functional change.
 1.950  11-Sep-2021  rillig make: eliminate a few common subexpressions

This commit contains only changes that don't increase the code size on
x86_64. Replacing the remaining instance of ModChain_ShouldEval would
either increase the code size or make the code harder to understand.

No functional change.
 1.949  06-Sep-2021  rillig make: fix typos in comments
 1.948  25-Aug-2021  rillig make: fix lint warnings on ilp32 platforms

The warnings in arch.c were about conversions from int to long.

The warnings in var.c were about conversions from long long to
ptrdiff_t.

No functional change.
 1.947  14-Aug-2021  rillig make: remove workaround for initialization bug in lint

The bug has been fixed in init.c 1.208 from 2021-08-14.
 1.946  08-Aug-2021  rillig make: remove redundant initialization in ApplyModifier_Order

No functional change.
 1.945  31-Jul-2021  rillig make: fix lint warnings

The string functions from str.h are declared as 'static __unused' when
compiled with GCC, but lint explicitly undefines __GCC__ during
preprocessing. Therefore, make those functions inline, to prevent
warnings that they are unused.

The macro UNCONST is used in a few places, and (again) since lint
undefines __GCC__, that macro expanded to a simple type cast, which lint
warned about. To prevent this warning, implement UNCONST as a function
that works everywhere and hides the type cast.

In filemon_open, the code for closing F->in was obviously unreachable.

No functional change.
 1.944  31-Jul-2021  rillig make: clean up ApplyModifier_Order

No functional change.
 1.943  30-Jul-2021  rillig make: reword comment for ApplyModifier_Order
 1.942  30-Jul-2021  rillig make: handle parse errors in ':O' uniformly

Previously, the error handling for the variable modifier ':O' differed
depending on the exact variant and in some cases led to misleading
or missing diagnostics.
 1.941  30-Jul-2021  rillig make: merge duplicate code for sorting strings and numbers

No functional change.
 1.940  30-Jul-2021  rillig make: fix typo in manual page, add more tests for the new ':On'
 1.939  30-Jul-2021  sjg Add :On for numeric sort

Reviewed by: christos rillig
 1.938  21-Jun-2021  rillig make: use more practical data type in RegexReplace

While size_t is most appropriate for array indexes, make needs to be
compatible with C90, which does not support the %zu printf conversion.
To avoid type casts, use a simple unsigned int here, which is more than
enough for storing a single decimal digit.

No functional change.
 1.937  21-Jun-2021  rillig make: extract RegexReplace from ModifyWord_SubstRegex

No functional change.
 1.936  21-Jun-2021  rillig make: move all labels in ModifyWord_SubstRegex to the same level

No functional change.
 1.935  21-Jun-2021  rillig make: reduce indentation in ModifyWord_SubstRegex for ':C'

No functional change.
 1.934  21-Jun-2021  rillig make: fix documentation for ParseModifierPart
 1.933  21-Jun-2021  sjg Do not report unmatched regex subexpressions

It is not always an error for a subexpression to have not matched,
since the regex library can/does not convey how many matches are
expected, only report an error if opts.strict (-dL)

Reviewed by: christos
 1.932  30-May-2021  rillig make: uncompress code in ApplyModifier_Unique

The algorithm is easier to understand when each line of code only
focuses on a single topic.

No change to the resulting binary, except for line numbers in assertion
messages.
 1.931  30-May-2021  rillig make: inline VarUniq into ApplyModifier_Unique

No functional change.
 1.930  19-Apr-2021  rillig branches: 1.930.2;
make: use straight quotes for error 'Bad conditional expression'

This diagnostic was supposed to be an error, see ApplyModifier_IfElse.

When such an error occurs while the makefiles are read, make stops with
an error, as can be expected. But when such an error occurs later,
after all makefiles have been read, the message is printed but make does
not stop.

In lint mode (-dL), make stops in such a case. I didn't dare to make
this the default behavior, out of fear of breaking existing build
infrastructure, not only in NetBSD or pkgsrc, but also FreeBSD and other
operating systems that use the bmake distribution, generated from the
same source code.
 1.929  14-Apr-2021  rillig make: reduce memory allocations in the modifiers ':D' and ':U'
 1.928  14-Apr-2021  rillig make: rename members of ModifyWord_LoopArgs

No functional change.
 1.927  14-Apr-2021  rillig make: clean up pattern flags for the modifiers ':S' and ':C'

No special handling is necessary for C90 since none of the struct
members is a const_member.

The prefix 'Var' is not necessary since this type does not apply to a
variable but only to a modifier.

No functional change.
 1.926  12-Apr-2021  rillig make: spell SysV in mixed case

No functional change.
 1.925  12-Apr-2021  rillig make: reduce memory allocation and strlen calls in modifier ':from=to'

Previously, SysVMatch was quite verbose and felt like hand-optimized
assembler code, which made it difficult to discover the underlying idea
of the code.

All this code was replaced with two simple calls to Substring_HasPrefix
and Substring_HasSuffix. Now that the operands of that modifier are no
longer passed as C strings, there is no need to collect all information
in a single scan through the word and the pattern.

It was not necessary to call Var_Subst unconditionally. Calling it only
when the string contains a '$' saves another memory allocation and two
string copies (because of the Buf_DoneDataCompact).

No functional change.
 1.924  12-Apr-2021  rillig make: in the ':Q' modifier, only allocate memory if necessary
 1.923  11-Apr-2021  rillig make: improve performance for LazyBuf

The previous O(n^2) time complexity for parsing a long string with many
variable expressions was not meant to last for long. I had hoped to fix
it within a few minutes, but that will take more time.

For now, make LazyBuf simpler by using a traditional C string for the
expected part instead of a Substring. This avoids a strlen call per
Var_Parse.

No functional change, only performance.
 1.922  11-Apr-2021  rillig make: remove redundant parameter from ParseVarnameLong

No functional change.
 1.921  11-Apr-2021  rillig make: migrate ParseModifierPart to use Substring

This will reduce memory allocation for modifier parts without the escape
characters '$' or '\'.

No functional change.
 1.920  11-Apr-2021  rillig make: avoid unnecessary calls to strlen when evaluating modifiers

No functional change.
 1.919  11-Apr-2021  rillig make: migrate ModifyWord functions to use Substring

This benefits the modifiers ':T' and ':H' since these scan the word from
the end. The SysV modifier '.c=.o' does not benefit yet, this will be
done in a follow-up commit.

Currently ModifyWords calls strlen for each single word, which degrades
performance. This will be cleaned up in a follow-up commit as well.

No functional change.
 1.918  11-Apr-2021  rillig make: migrate handling of the modifier ':S,from,to,' to Substring

Right now this does not gain any performance, it only makes the code in
ModifyWord_Subst a little simpler since it only uses
Buf_AddBytesBetween, not a mixture with Buf_AddBytes.

When the word passed to the ModifyWord functions is converted to
Substring as well, the call to strlen will become redundant.

No functional change.
 1.917  11-Apr-2021  rillig make: clean up remaining references to VarEvalFlags

VarEvalFlags has been replaced with VarEvalMode. There were some
comments and tests that still referred to the old names.

No functional change.
 1.916  11-Apr-2021  rillig make: avoid allocating memory for simple variable names

The main change is in ParseVarname, where a Buffer is replaced with the
newly introduced LazyBuf. LazyBuf is inspired by
https://golang.org/src/path/path.go.

In CanonicalVarname, the pre-comparison of the first letter of the
variable name is no longer necessary. GCC 9 optimizes a fixed-length
memcmp so well that the code can finally be written to target human
readers, leaving the optimization to the compiler.
 1.915  10-Apr-2021  rillig make: fix out-of-bounds read in Var_Parse_FastLane (since 30 minutes)

This bug made the test varmod-edge.mk fail sometimes with varying error
messages, as can be expected for an out-of-bounds read.
 1.914  10-Apr-2021  rillig make: reword debug log message for empty variable name

The function names did not match anymore, after one of the many
refactorings in the last few months.
 1.913  10-Apr-2021  rillig make: reduce debug logging and memory allocation for ${:U...}

Expressions of the form ${:U...} are often generated by .for loops.
Since these expressions are not generated knowingly by the make user, do
not fill the debug log with them since that would interrupt the normal
reading flow of the -dv log for nested expressions.
 1.912  06-Apr-2021  rillig make: reduce verbosity of the -dv debug logging for standard cases

The verbosity was already removed from LogBeforeApply, now it is
consistent between LogBeforeApply and LogAfterApply.
 1.911  05-Apr-2021  rillig make: in debug log, add space between scope and variable name

Without this space, the debug log looked more like line noise, even
though the only punctuation was a single innocent ':'. From a make
user's perspective, the variable name is a word of its own and should
not be visually glued to its namespace.
 1.910  05-Apr-2021  rillig make: omit unnecessary details from -dv debug log

When an expression is based on a defined variable, it does not matter
whether the evaluation mode is "eval" or "eval-defined", therefore omit
these details to reduce confusion.
 1.909  05-Apr-2021  rillig make: be more verbose in -dv debug logging

The previous log output was too brief to be understandable. Give more
hints by describing each part of the expression when evaluating a
modifier. Distinguish between parse-only mode and eval mode since in
parse-only mode most of the details are irrelevant.
 1.908  05-Apr-2021  rillig make: clean up debug logging for ':M' and ':N'

Using square brackets as quotes was confusing since patterns can contain
square brackets themselves.

The debug logging for VarMatch was a bit too detailed. Having the
"before" and "after" states is enough for all practically relevant
cases.
 1.907  04-Apr-2021  rillig make: disallow '$' in the variable name of the modifier ':@'

If this restriction should break any existing makefile, the author of
that makefile was probably heading for the IOMCC.
 1.906  04-Apr-2021  rillig make: convert VarEvalFlags back into an enum, but not a bit-set

As was apparent in VarEvalFlags_ToString, a bit-set was not the best
data type since most of the flags were not freely combinable. The two
flags that could be combined were keepDollar and keepUndef, but even
these have distinguished names in the debug log.

The downside of struct bit-fields is that they need extra helper
functions in C90 (see nonints.h). Exchange these for a few helper
functions in var.c, to keep the code outside var.c simple.

No functional change.
 1.905  04-Apr-2021  rillig make: extract decision for evaluating an expression

No functional change.
 1.904  03-Apr-2021  rillig make: move VarEvalFlags_ToString down to the other functions

No functional change.
 1.903  03-Apr-2021  rillig make: condense definition of VarPatternFlags

No functional change.
 1.902  03-Apr-2021  rillig make: inline VarFlags into Var

No functional change.
 1.901  03-Apr-2021  rillig make: replace enum bit-set with struct bit-fields

This makes the code easier to read, especially when setting one of the
flags to false.

No functional change.
 1.900  03-Apr-2021  rillig make: remove unused Expr.varFlags

No functional change.
 1.899  03-Apr-2021  rillig make: remove VarFlags from debug logging

Before the introduction of ExprDefined, VarFlags contained whether the
expression was defined or not, which was useful to know since the final
value of the expression depends on this information. The other VarFlags
do not influence the evaluation, so there is no point logging them.
 1.898  03-Apr-2021  rillig make: simplify access to the name of an expression during evaluation

During the evaluation, the flags of the variable don't change, except
for VFL_IN_USE, but in such a case make exits immediately anyway.

Logging the variable flags does not provide any useful information, so
it will be removed soon. Logging the variable flags made sense when the
definedness of the expression was still encoded in the variable flags,
instead of in the separate ExprDefined.

No functional change.
 1.897  03-Apr-2021  rillig make: backport to C90

In the past few months I had accidentally used C99 features in the make
code. According to tools/README, tools that are used in the build
system should restrict themselves to C90.

This allows make to build with GCC's options "-pedantic
-Wno-system-headers -Dinline= -Wno-error=cast-qual".

I didn't notice anyone actively complaining though, I just wanted to see
how much work this backporting would be. The identifier __func__ is
still used, as in other tools.

No functional change.
 1.896  03-Apr-2021  rillig make: rename function parameters to match boolean constants

No functional change.
 1.895  03-Apr-2021  rillig make: use C99 bool type instead of defining its own

No functional change.
 1.894  30-Mar-2021  rillig make: remove workaround for bug in lint

The bug has been fixed in lint.

No functional change.
 1.893  21-Mar-2021  rillig make: fix typos in documentation of ModChain
 1.892  16-Mar-2021  rillig make: fix documentation for ModChain

The outer ModChain can be interrupted by an inner ModChain, but it
continues to exist.

No functional change.
 1.891  15-Mar-2021  rillig make: rename ApplyModifiersState to ModChain

The new name accurately describes the structural element that holds such
properties as the separator character and whether the expression value
is considered a single word. The old name ApplyModifiersState was too
long and was meant as a placeholder anyway, when I introduced it in
var.c 1.236 from 2020-07-03.
 1.890  15-Mar-2021  rillig make: improve documentation of ApplyModifiersState

No functional change.
 1.889  15-Mar-2021  rillig make: document an example for a 'chain of modifiers'

No functional change.
 1.888  15-Mar-2021  rillig make: fix double varname expansion in the variable modifier '::='

This is an edge case that doesn't occur in practice since pretty much
nobody dares to use variable names that contain an actual '$' in their
name. This is not about the fairly common VAR.${param} (as written in
the makefile), but instead about the variable whose name is literally
'VAR.${param}'.

The test demonstrates that after the fix, the variable name is taken
exactly as-is for the simple assignment modifier '::='. There are no
such tests for the modifiers '::+=', '::!=' and '::?=', but that's ok.
The code in ApplyModifier_Assign would look assymetrical and suspicious
enough if one of these modifiers would expand its variable name and the
others wouldn't.
 1.887  15-Mar-2021  rillig make: clean up documentation of ApplyModifiersState

No functional change.
 1.886  15-Mar-2021  rillig make: change debug log for variable evaluation flags to lowercase

This makes them easier distinguishable from variable names since the
latter are usually uppercase.

No functional change outside debug mode.
 1.885  15-Mar-2021  rillig make: replace enum bit-field with struct bit-field for VarEvalFlags

This makes the code easier to read, especially in var.c. It also makes
debugging sessions easier since some debuggers don't show enum
bit-fields symbolically as soon as more than one bit is set.

The code outside var.c is basically unchanged, except that instead of
passing the individual flags, there are 4 predefined evaluation modes.
These suffice for all practical use cases. Only in the implementation
deep inside var.c, the value of the flags keepDollar and keepUndef
differs.

There is no way of passing the struct to EnumFlags_ToString, which means
the ToString function has to be spelled out explicitly. This allows for
fine-tuning the representation in the debug log, to reduce the amount of
uppercae letters.

No functional change.
 1.884  15-Mar-2021  rillig make: rename VARE_NONE to VARE_PARSE_ONLY

The name 'NONE' described the bit pattern, which was not useful to
understand its meaning. Omitting VARE_WANTRES only parses the
expression, without evaluating any part of it.

No functional change, not even in debug mode since Enum_FlagsToString
always returns "none" for all-bits-unset.
 1.883  14-Mar-2021  rillig make: fix documentation of VarFreeEnv

No functional change.
 1.882  14-Mar-2021  rillig make: skip variable lookup for '::=' modifiers in parse-only mode

This is just to keep the code consistent among the various variable
modifiers. The performance gain is negligible.

The actual assignment to the variable had already been skipped
previously.

No functional change.
 1.881  14-Mar-2021  rillig make: separate parsing and evaluating for modifier '::='

No functional change.
 1.880  14-Mar-2021  rillig make: in parse-only mode, do not update the expression value in ':sh'

No functional change outside debug mode. The other variable modifiers
behave in the same way.
 1.879  14-Mar-2021  rillig make: do not evaluate the ':O' modifiers in parse-only mode

No functional change in practical usage. Theoretically this change can
be observed by looking at the generated random numbers for the ':Ox'
modifier, but the quality or exact sequence of these random numbers is
not guaranteed anyway.
 1.878  14-Mar-2021  rillig make: separate parsing and evaluating in the ':O' modifiers

No functional change.
 1.877  14-Mar-2021  rillig make: do not evaluate modifier ':[...]' in parse-only mode

In parse-only mode, variable expressions in the argument to that
modifier are not resolved. This led to the error message about the 'Bad
modifier' in var-eval-short.mk.
 1.876  14-Mar-2021  rillig make: do not evaluate modifiers ':M' and ':N' in parse-only mode

No functional change outside debug mode (-dv for ModifyWord_Match).
 1.875  14-Mar-2021  rillig make: do not return unevaluated 'else' part from the ':?' modifier

No functional change outside debug mode.
 1.874  14-Mar-2021  rillig make: do not evaluate ':gmtime' and ':localtime' in parse-only mode

No functional change.
 1.873  14-Mar-2021  rillig make: don't evaluate several simple modifiers in parse-only mode

This affects the modifiers ':E', ':H', ':P', ':Q', ':R', ':T', ':hash',
':q', ':range', ':tl', ':ts', ':tu', and ':u'. All these modifiers are
side-effect free.

Skipping the evaluation for these modifiers is purely for code
consistency and performance.

No functional change.
 1.872  14-Mar-2021  rillig make: skip strdup when parsing an irrelevant ':L' modifier

No functional change.
 1.871  14-Mar-2021  rillig make: skip memcpy when parsing but not evaluating ':D' and ':U'

No functional change, just a tiny bit of performance improvement,
probably not even measurable. Having the code nevertheless serves as a
copy-and-paste template for implementing other modifiers that might
perform more costly tasks.
 1.870  14-Mar-2021  rillig make: only evaluate the ':@' modifier if the result is actually used

The test 'var-eval-short' had produced the output 'unexpected' before,
on stderr. It had been generated by '${:Uword:@${FAIL}@expr@}' by
combining the following obscure "features" of make:

1. the ':@' modifier loops over the words of the variable. This
modifier is not really obscure, it still takes some time to get used
to it.

2. the ':@' modifier allows a '$' sign in the variable name, which is
useless in practice.

3. the ':@' modifier creates a temporary loop variable in the global
namespace. Luckily there are only few collisions with other
variable names since their naming conventions differ.

4. after looping over the words of the expression, the temporary global
loop variable is deleted, and at that point the '$' is expanded,
being interpreted as the start of a variable expression.

5. The ':@' modifier deleted the global variable even when it was
called in parse-only mode (without VARE_WANTRES).

When the modifier ':@' was initially added to make in var.c 1.40 from
2000-04-29, Var_Delete didn't expand the variable name. That feature
was added in var.c 1.174 from 2013-05-18, probably without thinking of
this very edge-casey combination of features.

This commit fixes item 5 from the above list. The other obscurities
remain for now.
 1.869  14-Mar-2021  rillig make: eliminate common subexpression in ApplyModifier_Remember

No functional change.
 1.868  14-Mar-2021  rillig make: merge duplicate code in ApplyModifier_Remember

This way, parsing and evaluating the modifier is only written once in
the code. The downside is that the variable name is allocated even if
VARE_WANTRES is not set, but since this modifier is so obscure and
seldom used this doesn't matter in practice.
 1.867  14-Mar-2021  rillig make: do not expand the variable name in the ':_' modifier

This edge case had been so obscure that even discovering this takes
quite some time and requires reading the source code of make.

The manual page doesn't document whether the variable name is expanded
or not, it doesn't even give an example. When this obscure modifier was
initially added in var.c 1.210 from 2017-01-30, Var_Set always expanded
the variable name once, and there was no way around it. Therefore this
expansion has probably been unintentional.
 1.866  14-Mar-2021  rillig make: only evaluate the ':_' modifier if the expression is needed

See var-eval-short.mk:46 for the test demonstrating this change.
Previously, the expression ${:Uword:_=VAR} was evaluated including all
its side effects even though it was in an irrelevant branch of the
condition.
 1.865  14-Mar-2021  rillig make: return failure in TryParseIntBase0 for empty string

No functional change since the only caller of TryParseIntBase0 already
handles all possible parse errors. Without this check, the code just
looked wrong though.
 1.864  14-Mar-2021  rillig make: separate parsing from evaluating for several modifiers

This aligns the implementation of these modifiers with the requirements
in the long comment starting with 'The ApplyModifier functions'.

No functional change.
 1.863  14-Mar-2021  rillig make: reduce indentation in ApplyModifier_SunShell

No functional change.
 1.862  14-Mar-2021  rillig make: separate parsing and evaluating for modifier ':u'

No functional change.
 1.861  14-Mar-2021  rillig make: separate parsing and evaluating for modifiers ':Q' and ':q'

No functional change.
 1.860  14-Mar-2021  rillig make: extract duplicate code for parsing ':S' and ':C'

No functional change.
 1.859  14-Mar-2021  rillig make: extract duplicate code for detecting a modifier's end

No functional change.
 1.858  14-Mar-2021  rillig make: document how error handling should be done correctly

Right now, when a variable expression cannot be parsed, the result of
calling Var_Subst is a string containing garbage, and no error is
reported. In addition, there are some silent errors that are not
reported at all. This combination makes it difficult to change the
error handling without introducing subtle breakage in some edge cases.

An example for garbage output is in varmod-subst-regex.mk, in target
mod-regex-compile-error.

No functional change.
 1.857  14-Mar-2021  rillig make: sort modifiers in ApplyModifier alphabetically

No functional change.
 1.856  14-Mar-2021  rillig make: fix wrong expression evaluation in -dL mode

The modifier ':C' now only compiles the regular expression if the result
of the expression is actually needed.

Several other modifiers have the same bug of evaluating the expression
in cases where this is not needed. It just doesn't show up because they
don't have any noticeable side effects, other than wasting CPU time.
This affects irrelevant conditions as well.
 1.855  23-Feb-2021  rillig make: reduce indentation in Var_Parse

The extra condition had been necessary before FStr made memory
management simpler.

The Coverity annotation got out-of-date when the parameter was converted
to FStr since that type is not allocated on the heap, only its inner
members are.

No functional change.
 1.854  23-Feb-2021  rillig make: restructure code in ParseVarname to target human readers

Breaking the loop once for depth == 0 and once for depth == 1 was
unnecessarily confusing, as was the nested 'if'. Start counting with 0
since there is no reason to start at 1.

Evaluating the common subexpression '*p == endc' is left as an exercise
to the compiler.

No functional change.
 1.853  23-Feb-2021  rillig make: improve error message for bad modifier in variable expression

The improvement is especially noteable for variable expressions based on
the empty variable, see moderrs.exp:103.
 1.852  23-Feb-2021  rillig make: improve error message for unclosed modifier

Replace "variable specification" with the more modern "variable
expression", reduce the number of parentheses, output more than a single
character for modifiers, make it obvious that in expressions such as
${:Serror}, the "" means a variable name.
 1.851  23-Feb-2021  rillig make: improve error message for unknown modifier

Back in 1995, the modifiers were all single-character, and it made sense
to print only the first character. Nowadays, with ':S', ':@var@...@',
'::=' and several others, a little more context is useful to see where
the exact error is. The actual modifier is still guessed, and the guess
may be wrong as soon as backslashes get involved, but it is still better
than before.
 1.850  23-Feb-2021  rillig make: remove redundant parameter of ApplySingleModifier
 1.849  23-Feb-2021  rillig make: explain non-obvious code around indirect variable modifiers

No functional change.
 1.848  23-Feb-2021  rillig make: comment on possible inconsistency in handling modifier ':sh'
 1.847  23-Feb-2021  rillig make: note inconsistent parsing behavior in modifier ':_='
 1.846  23-Feb-2021  rillig make: quote ':S' in error message about missing delimiter
 1.845  23-Feb-2021  rillig make: extract ParseModifier_Match into separate function

No functional change.
 1.844  23-Feb-2021  rillig make: add context information to error message about ':range' modifier
 1.843  23-Feb-2021  rillig make: fix local variable name for parsing arguments

The variable name 'arg' was misleading since after a successful
TryParseTime, it would no longer point to the argument of the variable
modifier, but to the _end_ of the argument. To reduce confusion, use p
instead, like everywhere else. This name is less specific, which is
still better than a wrong name.
 1.842  23-Feb-2021  rillig make: add quotes around variable name in an error message
 1.841  22-Feb-2021  rillig make: reorder code in ModifyWords

No functional change.
 1.840  22-Feb-2021  rillig make: use more common parameter order for VarSelectWords

No functional change.
 1.839  22-Feb-2021  rillig make: make ModifyWord_Subst a little easier to understand

Addition is easier than subtraction, and the expression 'word + wordLen'
obviously means 'the end of the word', which was not as easy to spot
before.

No functional change.
 1.838  22-Feb-2021  rillig make: remove freestanding freeIt variables

These variables all belong to a string variable. Connect them using
FStr, which reduces the number of variables to keep track of.

No functional change.
 1.837  22-Feb-2021  rillig make: remove boolean parameter and return type from VarFreeEnv

There was only a single case where this parameter was false. Inline
that case. That was the only case that needed the return value, so
remove that as well.
 1.836  22-Feb-2021  rillig make: add a few open questions to Var_SetWithFlags
 1.835  22-Feb-2021  rillig make: do not expand variable name from the command line twice in a row

When

1. there is a global variable containing a dollar in its expanded name
(very unlikely since there are lots of undocumented edge cases that
make variable names containing dollar signs fragile), and

2. after that (unlikely since that requires .MAKEFLAGS instead of a
normal command line)

3. there is a command line variable of the same name (again very
unlikely since that variable name would contain a dollar sign as
well in the expanded form),

the global variable would not be undefined as promised by the comments
since its name was expanded once more than intended.

Because of the two 'very unlikely' above, this edge case hopefully does
not affect any practical use cases.

Note that this is not about VAR.${param} (which has a dollar sign in its
unexpanded form), but about the case where param itself would expand to
a dollar sign, such as after param=$$.
 1.834  22-Feb-2021  rillig make: extract ExistsInCmdline from Var_SetWithFlags

No functional change.
 1.833  22-Feb-2021  rillig make: save a hash map lookup when defining a cmdline variable

This is a preparation to extract the code for exporting a cmdline
variable. That code differs in several details from the other code in
ExportVar.

No functional change.
 1.832  22-Feb-2021  rillig make: document interaction between cmdline and global variables

Make prevents global variables from being or becoming visible when a
command line variable of the same name is already defined.

There is a double safety net here. Even if the call to Var_DeleteExpand
were removed, there would be no noticeable effect, other than one less
line in the debug log.

No functional change.
 1.831  16-Feb-2021  rillig make: clean up VarAdd, Var_Delete, Var_ReexportVars

No functional change.
 1.830  16-Feb-2021  rillig make: clean up and update comments in var.c

During the refactorings of the last months, several comments have become
outdated, some are now redundant since the code is as clear as the
comment, and some code benefits from a bit of explanation.
 1.829  16-Feb-2021  rillig make: use bit-shift expressions for VarFlags constants

These are easier to read than hex constants.

There was no need to skip bits 2 and 3 (there were no constants for 0x04
and 0x08). Close this gap, to avoid confusing future readers. Keep the
relative order of the flags since that affects the debug output of -dv.

No functional change.
 1.828  16-Feb-2021  rillig make: rename constants for VarFlags

The old prefix was "VAR_" and this prefix is used for several other
constants as well, which made it ambiguous.
 1.827  16-Feb-2021  rillig make: sync comment about duplicated code with reality

There are only very few places in var.c that contain really duplicate
code anymore.

There is still lots of _almost_ duplicate, for example the code for
parsing variable modifiers. It differs subtly in behavior:

* The modifiers ':M' and ':N' use '$$' to escape a '$' sign, while
almost all other modifiers use '\$' for this purpose.

* The modifiers ':M', ':N', ':S', ':@' and several others parse
balanced parentheses and braces, allowing '(' to '}' to match.
The modifiers ':D' and ':U' only treat the end character special but
not the other 3 of '(){}'.

* When parsing the modifier ':S' but not evaluating it, the code for
nested variable expressions is parsed differently from when it is in
evaluation mode (VARE_WANTRES). This applies to an outer ':S'
modifier and an inner ':D' or ':M' modifier.

Since these inconsistencies affect the behavior in edge cases and some
users of make might depend on it, they cannot be fixed by
behavior-preserving refactorings.
 1.826  15-Feb-2021  rillig make: rename ExprDefined constants for debug logging
 1.825  15-Feb-2021  rillig make: rename ExprStatus to ExprDefined

The type describes the definedness of an expression, not a general
status, therefore the new name is more precise.

The constants are renamed as well since their prefix 'VES' does not
match the type name anymore, it was correct 3 days ago when the type was
still named VarExprStatus. The name VES_NONE was misleading since
'none' does not describe its actual effect. That name came from the
time when the status was a bit set, and 'none' simply meant 'none of the
bits are set'.

The names used in debug logging will be renamed in a follow-up commit,
to demonstrate that the changes in this commit indeed have no functional
change, especially not the change from '!=' to '==' in line 4304.

No functional change.
 1.824  15-Feb-2021  rillig make: update comments for Expr, amending the previous commit
 1.823  15-Feb-2021  rillig make: split parameters for evaluating variable expressions

The details of how variable expressions are evaluated is controlled by
several parameters: startc and endc differ for $(VAR) and ${VAR}, the
value of the expression can be interpreted as a single big word, and
when joining several words (such as with ':M' or ':S'), there may be a
custom word separator (defined with ':ts*').

The scope of half of these parameters is the whole variable expression,
the other half of the parameters are reset after each chain of indirect
modifiers. To make this distinction obvious in the code, extract Expr
from ApplyModifiersState. Previously, these details were hidden in how
parameters are passed and restored among ApplyModifiersIndirect and
ApplyModifiers.

The changes in the individual ApplyModifier functions are numerous but
straight-forward. They mostly replace 'st' with 'expr'.

The changes in ApplyModifiers and ApplyModifiersIndirect are more
subtle. The value of the expression is no longer passed around but is
stored in a fixed location, in Expr, which makes it easier to reason
about memory management.

The code in ApplyModifiers after 'cleanup' looks quite different but
preserves the existing behavior. Expr_SetValueRefer is nothing else
than the combination of FStr_Done followed by FStr_InitRefer. Storing
exprStatus back at the end was responsible for passing the definedness
of the expression after applying the indirect modifiers back to the
outer ApplyModifiersState. The same effect is now achieved by having
Expr.status with a wider scope.

No functional change.
 1.822  15-Feb-2021  rillig make: clean up code and comments around ModifyWord

In ModifyWords, there is no "passed string" anymore since that function
now directly operates on the expression value.

While here, improve the documentation of ModifyWordsCallback and rename
it to ModifyWordProc, focusing on its purpose instead of where it is
used.
 1.821  14-Feb-2021  rillig make: clean up memory management in evaluation of expressions

The condition "st->newValue.str != val" in ApplySingleModifier made the
memory management look more complicated than it really was. Freeing an
object based on another object's value is harder to understand than
necessary.

To fix this, the "current value" of the expression is now stored in
ApplyModifiersState, and it gets updated in-place by the ApplyModifier
functions. This reduces the number of parameters for the ApplyModifier
functions.

Accessing the current value of the expression is now more verbose than
before (st->value.str instead of the simple val). To compensate for
this verbosity, ApplyModifiersIndirect is now much easier to understand
since there is no extra "current value" floating around.

There is still room for improvement. In ApplyModifiers, passing an FStr
in and returning another (or possibly the same) makes it difficult to
understand memory management. Adding a separate Expr type that outlives
the ApplyModifiersState will make this easier, in a follow-up commit.
 1.820  14-Feb-2021  rillig make: reduce redundant code around ModifyWords

The result of ModifyWords had been passed to Expr_SetValueOwn in all
cases. The last argument to ModifyWords had always been st->sep.
 1.819  14-Feb-2021  rillig make: print error about failed shell command before overwriting variable

Memory management of the value of variable expressions is currently more
complicated than necessary. It is the responsibility of ApplyModifiers,
even though conceptually the value belongs to an expression, so it
should rather be in Expr. Right now, this is an alias for
ApplyModifiersState, but that will change soon.

When that is done, there will no longer be a "current value" and a "new
value", only a single "value" of an expression. At that point, before
Expr_SetValueOwn will overwrite the old value with the output of the
shell command, the error message needs to refer to the latter.
 1.818  14-Feb-2021  rillig make: clean up ValidShortVarname

The switch statement was hard to read, especially the "break" that
needed a comment since it was effectively a "continue".
 1.817  14-Feb-2021  rillig make: clean up ParseVarnameShort

Single-character short variable expressions such as $V neither have a
starting character nor an ending character. The only interesting
character forms the complete variable name.

No functional change.
 1.816  14-Feb-2021  rillig make: rename VarExprStatus to ExprStatus

It is only used in var.c so there is no need to use a prefix.
 1.815  14-Feb-2021  rillig make: revert part of previous commit

That code was not ready yet.
 1.814  14-Feb-2021  rillig make: update line numbers in expected test output

The documentation from the previous commit added a few lines.
 1.813  14-Feb-2021  rillig make: add functions for assigning the value of an expression

The plan is to have only the "current value" of the expression as a
member, not the "new value". To do this consistently and get the memory
management right, there must be a single place (or two) where the value
of the expression is updated.

No functional change.
 1.812  14-Feb-2021  rillig make: rename ApplyModifiersState_Define to Expr_Define

The type name ApplyModifiersState was only intended as a working draft,
its name is too long and its scope a little too narrow.

Applying the modifiers is the main part of evaluating a variable
expression, and the scope of that type will be extended to parsing the
name of the expression as well. This will hopefully reduce the number
of parameters, which is currently at 14.

No functional change.
 1.811  14-Feb-2021  rillig make: condense the code for parsing :S and :C modifiers

No functional change.
 1.810  14-Feb-2021  rillig make: rearrange some comments to make them easier to spot
 1.809  14-Feb-2021  rillig make: document purpose of stepping back in the parser

In ApplyModifier_Assign there was no need to compute the delimiter from
st->startc since that has already be done at that point.
 1.808  06-Feb-2021  rillig make: sync comment for ApplyModifier_Assign with reality

In compat mode, empty shell commands are handled correctly since at
least 1993.

In jobs mode, empty shell commands are handled correctly since at least
job.c 1.93 from 2005-06-16.

The only place where empty shell commands led to problems was in
Cmd_Exec, but that has nothing to do with the example from the comment.
See var-op-shell.mk for more details.
 1.807  05-Feb-2021  rillig make: improve documentation about variable scopes

In an experiment, I tried to separate the concepts of a GNode and a
variable scope. The global variables SCOPE_GLOBAL, SCOPE_INTERNAL and
SCOPE_CMDLINE are implemented as GNode even though they only need the
members 'name' and 'vars'. All their other members are unused.
Therefore it seemed natural to extract this part of the GNode into a
separate type called Scope.

The resulting code was harder to read though since it had split the
namespace of the functions into several parts that were not obviously
related: The Var_ functions, the Scope_ functions, and the short-cut
Global_ functions. Because of this, I threw away the experiment.

All that is left are a few updated comments.
 1.806  05-Feb-2021  rillig make: rename remaining ctx to scope

One less small inconsistency. Everywhere else, that variable had been
named ctxt, not ctx.
 1.805  05-Feb-2021  rillig make: in the Var_ functions, move the scope to the front

This change provides for a more natural reading order in the code.
Placing the scope first makes it immediately clear in which context the
remaining parameters are interpreted.

No functional change.
 1.804  05-Feb-2021  rillig make: add shortcut Global_Delete for deleting a global variable
 1.803  04-Feb-2021  rillig make: rename Var_ValueDirect to GNode_ValueDirect
 1.802  04-Feb-2021  rillig make: rename context and ctxt to scope

This continues the previous commit, in which VAR_GLOBAL, VAR_INTERNAL
and VAR_CMDLINE were renamed.

Renaming the variable 'ctxt' was trivial since that word is used nowhere
else. In the comments though, each occurrence of the word 'context' had
to be checked individually since the word 'context' was not only used
for referring to a variable scope. It is also used to distinguish
different situations where characters are escaped in a certain way
('parsing context') and in a few other expressions.
 1.801  04-Feb-2021  rillig make: rename some VAR constants to SCOPE

The word "context" does not fit perfectly to the variables that are
associate with a GNode, as the context is usually something from the
outside and the variables are more like properties inherent to the
GNode.

The term "global context" fits even less. Since the thing where
variables are looked up is commonly named a scope, use that term
instead.

This commit only renames the global variables VAR_GLOBAL, VAR_INTERNAL
and VAR_CMDLINE, plus a few very closely related comments. These are:

GNode.vars (because of line breaks)
GNode_Free (dito)
varname-make_print_var_on_error.mk
varname-make_print_var_on_error-jobs.mk

The debug message in Var_Stats is left as-is since there is no unit test
for it yet.

The other renamings (variable names "context", "ctxt", as well as
further comments) will be done in a follow-up commit.
 1.800  04-Feb-2021  rillig make: merge SetVar into Var_SetWithFlags

No functional change.
 1.799  04-Feb-2021  rillig make: rename Var_SetWithFlags to Var_SetExpandWithFlags

Add back Var_SetWithFlags for the one call that doesn't need to expand
the name.

Now one of the flags is encoded in the function name while the others
are encoded in VarSetFlags. This is inconsistent. Maybe there is a
better way to model the different variants of setting a variable.
 1.798  04-Feb-2021  rillig make: rename Var_Set to Var_SetExpand

After doing the textual renaming across all files, I added a new
function Var_Set that does not expand the variable name. I then undid
the renaming for all calls where the variable name cannot ever contain a
dollar sign. I omitted the word "Expand" from the textual references in
the unit tests and in the debug logging messages since the focus is
usually on the "Set" part, not on the "Expand".

No functional change.
 1.797  03-Feb-2021  rillig make: rename Var_Delete to Var_DeleteExpand, Var_DeleteVar to Var_Delete

The function names now follow the naming scheme from the other functions
that handle variables.

There are several calls that remain syntactically unchanged but that
omit the call to strchr('$') now. Since all these calls use constant
variable names, there is no functional change.
 1.796  03-Feb-2021  rillig make: split Var_Exists into plain Var_Exists and Var_ExistsExpand

Most previous calls to Var_Exists use constant variable names. Only the
two calls in parse.c need to expand the variable name.

It may be a good idea to expand the variable name once in VarAssign_Eval
instead of repeating the expansion in each of its special cases.

No functional change.
 1.795  03-Feb-2021  rillig make: replace Global_AppendExpand with Global_Append

All callers with a variable name that is guaranteed to not contain a
dollar sign have been converted to call Global_Append instead of the
previous Global_AppendExpand. After that, Global_AppendExpand was
unused, therefore it was effectively just renamed.
 1.794  03-Feb-2021  rillig make: split Var_Append into Var_Append and Var_AppendExpand

The plain Var_Append now does not expand the variable name anymore. It
is used in situations where the variable name is known to not contain a
dollar sign.

This is a preparation for adding Global_Append, corresponding to
Global_AppendExpand.
 1.793  03-Feb-2021  rillig make: fix double expansion when appending to a new variable
 1.792  03-Feb-2021  rillig make: replace Global_SetExpand with Global_Set for constant names
 1.791  03-Feb-2021  rillig make: use shortcut functions Global_SetExpand and Global_AppendExpand

There are many places where global variables are set or appended to. To
reduce clutter and code size, encode the VAR_GLOBAL in the function
name.

The word Expand in the function names says that the variable name is
expanded. In most of the cases, this is not necessary, but there are no
corresponding functions Global_Set or Global_Append yet.

Encoding the information whether the name is expanded or not in the
function name will make inconsistencies obvious in future manual code
reviews. Letting the compiler check this by using different types for
unexpanded and expanded variable names is probably not worth the effort.
There are still a few bugs to be fixed, such as in SetVar, which expands
the variable name twice in a row.
 1.790  02-Feb-2021  rillig make: remove unused INTERNAL flag

It had been used for cached_realpaths, until this variable had its type
changed from GNode to HashTable in main.c 1.469 from 2020-11-14.
 1.789  02-Feb-2021  rillig make: replace VarExprFlags with VarExprStatus

The combination of !VEF_UNDEF && VEF_DEF was not possible, which made it
rather strange to model this state as a bit set.

The only functional change is the renamed constants in the debug output.

Using ENUM_VALUE_RTTI_2 felt like overengineering since it's harder to
understand than a simple array of names.
 1.788  02-Feb-2021  rillig make: replace enum bit-set with bit-field

No functional change.

The generated code from GCC 5.5 is very similar. On x86_64, memory
access is no longer in 32-bit units but in 8-bit units since only the
first few bits are actually used. The bit patterns are the same as
before, so if there is any difference in performance, GCC should have
chosen the more efficient variant all along.

In a previous experiment, the code size increased a lot, surprisingly.
 1.787  01-Feb-2021  rillig make: replace pre-increment with post-increment or simple addition

The rest of the code already prefers post-increment if there is no
actual difference.
 1.786  30-Jan-2021  rillig make(1): inline Buf_Len
 1.785  30-Jan-2021  rillig make(1): inline Buf_GetAll
 1.784  30-Jan-2021  rillig make(1): split Buf_Destroy into Buf_Done and Buf_DoneData

In all cases except one, the boolean argument to Buf_Destroy was
constant. Removing that argument by splitting the function into two
separate functions makes the intention clearer on the call site. It
also removes the possibility for using the return value of Buf_Done,
which would have made no sense.

The function Buf_Done now pairs with Buf_Init, just as in HashTable and
Lst.

Even though Buf_Done is essentially a no-op, it is kept as a function,
both for symmetry with Buf_Init and for clearing the Buffer members
after use (this will be done only in CLEANUP mode, in a follow-up
commit).
 1.783  30-Jan-2021  rillig make(1): reduce boilerplate for printing bit sets in debug mode

No functional change.
 1.782  16-Jan-2021  rillig make(1): fix a few inconsistencies for lint's strict bool mode
 1.781  10-Jan-2021  rillig make(1): make a few more bool expressions more precise

The previous version of lint(1) from a few hours ago didn't catch all
occurrences. And even the current one doesn't catch everything.
Function arguments and return types still need some work. The "return
quietly" from shouldDieQuietly still implicitly converts from int to
_Bool.

No functional change.
 1.780  10-Jan-2021  rillig make(1): consistently use boolean expressions in conditions

Most of the make code already followed the style of explicitly writing
(ptr != NULL) instead of the shorter (ptr) in conditions.

The remaining 50 instances have been found by an experimental,
unpublished check in lint(1) that treats bool expressions as
incompatible to any other scalar type, just as in Java, C#, Pascal and
several other languages.

The only unsafe operation on Boolean that is left over is (flags &
FLAG), for an enum implementing a bit set. If Boolean is an ordinary
integer type (the default), some high bits may get lost. But if Boolean
is the same as _Bool (by compiling with -DUSE_C99_BOOLEAN), C99 6.3.1.2
defines that a conversion from any scalar to the type _Bool acts as a
comparison to 0, which cannot lose any bits.
 1.779  09-Jan-2021  rillig make(1): fix lint warnings
 1.778  30-Dec-2020  rillig make(1): format multi-line comments
 1.777  29-Dec-2020  rillig make(1): clean up code in extracted ExportVar functions
 1.776  29-Dec-2020  rillig make(1): split ExportVar into separate functions
 1.775  29-Dec-2020  rillig make(1): rename local variables in Var_ReexportVars
 1.774  28-Dec-2020  rillig make(1): replace global preserveUndefined with VARE_KEEP_UNDEF

Controlling the expansion of variable expressions using a global
variable and a VARE flag was inconsistent.

Converting the global variable into a flag had to prerequisites:

1. The unintended duplicate variable assignment had to be fixed, as
done in parse.c 1.520 from 2020-12-27. Without this fix, it would have
been necessary to add more flags to Var_Exists and Var_SetWithFlags, and
this would have become too complex.

2. There had to be a unit test demonstrating that VARE_KEEP_DOLLAR only
applies to the top-level expression and is not passed to the
subexpressions, while VARE_KEEP_UNDEF applies to all subexpressions as
well. This test is in var-op-expand.mk 1.10 from 2020-12-28, at least
for the ':@word@' modifier. In ParseModifierPartSubst, VARE_KEEP_UNDEF
is not passed down either, in the same way.
 1.773  27-Dec-2020  rillig make(1): remove dead code from ApplyModifiersIndirect

At that point, the expression can never be varUndefined. At the
beginning of ParseVarnameLong, the expression is initialized to a simple
empty string, and that string is only ever converted to varUndefined at
the very end of Var_Parse.
 1.772  27-Dec-2020  rillig make(1): remove outdated comment about string comparisons

Back in 1993, the variables in a context were stored in a linked list.
Searching such a list indeed required literally thousands of calls to
strcmp. In make.h 1.22 from 1999-09-15, the linked list was replaced
with a hash table, requiring much fewer string comparisons. Since then,
the rationale doesn't apply anymore.
 1.771  27-Dec-2020  rillig make(1): remove unnecessary VPR_ERR_SILENT
 1.770  27-Dec-2020  rillig make(1): do not inspect output variables in ParseVarnameShort
 1.769  27-Dec-2020  rillig make(1): move error handling code out of UndefinedShortVarValue
 1.768  27-Dec-2020  rillig make(1): split Var_Subst into easily understandable functions

Extracting the character-level details makes the essence of Var_Subst
visible in the code, which is to iterate over the given text, handling a
few types of tokens.
 1.767  27-Dec-2020  rillig make(1): clean up VarParseResult constants

The many constants were invented because at that time I didn't quite
understand the actual outcomes of Var_Parse that need to be
distinguished. There are only a few:

(1) Errors, whether they are parse errors, or evaluation errors or
undefined variables. The old constants VPR_PARSE_MSG and
VPR_UNDEF_MSG are merged into VPR_ERR.

(2) Undefined expressions in a situation in which they are allowed.
Previously the documentation for VPR_UNDEF_SILENT talked about
undefined expressions in situations where they were not allowed.
That case is fully covered by VPR_ERR instead.

(3) Errors that are silently ignored. These are probably bugs.

(4) Everything went fine, the expression has a defined value.
 1.766  27-Dec-2020  rillig make(1): remove unnecessary VPR_UNKNOWN for error handling

There is no sensible way for a caller of Var_Parse to deal with an error
state of "maybe successful, maybe not", therefore remove the constant
for it.
 1.765  27-Dec-2020  rillig make(1): align names of VarExportMode with the directives
 1.764  23-Dec-2020  rillig make(1): rename CmdOpts.lint to strict

When running lint(1) on the code, it defines the preprocessor macro
"lint" to 1, which generated a syntax error in the declaration "Boolean
lint", as that became "Boolean 1".
 1.763  23-Dec-2020  rillig make(1): fix memory leak in Var_Undef (since 2020-12-22)
 1.762  22-Dec-2020  rillig make(1): allow .undef to undefine multiple variables at once

Since make doesn't support variable names containing spaces, this edge
case is not enough reason to stop this feature. Having multiple
variable names as arguments nicely aligns with other directives such as
.for and .export.
 1.761  21-Dec-2020  rillig make(1): remove excess newline from parse errors (since 2020-11-01)

For the modifiers :gmtime and :localtime, the excess newline had been
added in var.c 1.631 from 2020-10-31 21:40:20.

For the modifiers :range and :ts, the excess newline had been added in
var.c 1.635 from 2020-11-01 14:36:25.
 1.760  21-Dec-2020  rillig make(1): remove redundant parameters from ParseModifierPart
 1.759  21-Dec-2020  rillig make(1): save a few memory allocations in variable expressions
 1.758  21-Dec-2020  rillig make(1): switch memory handling from MFStr to FStr

This makes all intermediate strings constant. For this simple
search-and-replace refactoring, all intermediate locations where the
"current value of the expression" was stored had to be of the type
MFStr.

Using FStr instead of MFStr allows to save a few memory allocations,
which will be done in the follow-up commits.
 1.757  21-Dec-2020  rillig make(1): clean up memory management for expanding variable expressions

Previously, memory management had been split among several variables.
The general idea was very simple though. The current value of the
expression needs to be kept in memory, and each modifier either keeps
that value or replaces it with its own newly allocated result, or
var_Error or varUndefined.

Using MFStr, it does not matter anymore that var_Error and varUndefined
are statically allocated since these are assigned using MFStr_InitRefer.

The complexity of the implementation is now closer to the actual
complexity. Most probably the code can be simplified even more.
 1.756  20-Dec-2020  rillig make(1): use FStr for ApplyModifiersState.newVal

Memory management is still complicated in this area. To clean this up,
the previous value of the expression needs to be converted to an MFStr
first, and later to an FStr.
 1.755  20-Dec-2020  rillig make(1): remove redundant assignment in ApplyModifier_SysV
 1.754  20-Dec-2020  rillig make(1): error out on unknown variable modifiers at parse time

Before, make printed an "error message" that did not include the word
error and thus was not easily identified as such. This "error message"
also did not influence the exit status in the default mode but only in
-dL mode. The error message also didn't include any line number
information and was thus rude.
 1.753  20-Dec-2020  rillig make(1): remove wrong error message for indirect modifier in lint mode
 1.752  20-Dec-2020  rillig make(1): document that indirect modifiers fall back to SysV modifiers
 1.751  20-Dec-2020  rillig make(1): remove redundant const declarations for parameters
 1.750  20-Dec-2020  rillig make(1): extract ApplySingleModifier from ApplyModifiers
 1.749  20-Dec-2020  rillig make(1): turn ApplyModifiersState.val into a local variable

This reduces the scope and makes it more obvious at which places this
variable can be changed and how the memory management is done.
 1.748  20-Dec-2020  rillig make(1): use FStr for memory management in Var_SetWithFlags
 1.747  20-Dec-2020  rillig make(1): extract SetVar from Var_SetWithFlags

SetVar contains the part that is not concerned about memory management
and expansion of the variable name.
 1.746  20-Dec-2020  rillig make(1): fix memory leak in Var_Delete (since yesterday)

The memory leak had been introduced in var.c 1.736 from 2020-12-19
20:47:24.
 1.745  20-Dec-2020  rillig make(1): change return type of Var_Value to FStr
 1.744  20-Dec-2020  rillig make(1): use FStr in VarNew
 1.743  20-Dec-2020  rillig make(1): return FStr from Var_Parse

This reduces the number of variable declarations at the call sites.
 1.742  20-Dec-2020  rillig make(1): export FStr and MFStr

These types are a replacement for the pattern "var + var_freeIt" that
enforces these two variables to be updated together.
 1.741  20-Dec-2020  rillig make(1): document memory handling in ApplyModifiers
 1.740  20-Dec-2020  rillig make(1): replace freeIt with FStr in EvalUndefined

Previously, the parameter out_freeIt was not guaranteed to be
initialized in every case, at least when looking only at EvalUndefined.
This contradicted the variable name.

Replace the two parameters with a single FStr to ensure that these
variables are always initialized together.
 1.739  20-Dec-2020  rillig make(1): replace FStr_Assign with separate initialization functions

In GetVarnamesToUnexport, there is no need to free the local FStr since
the only place where it is assigned an allocated string is at the very
end.

Having separate functions for the two main use cases of a possibly
allocated string makes the calling code simpler. This is a preparatory
commit for making the memory allocation in ApplyModifiers easier to
understand.
 1.738  20-Dec-2020  rillig make(1): extract string functions from ApplyModifier_To
 1.737  19-Dec-2020  rillig make(1): error out if .undef has not exactly 1 argument
 1.736  19-Dec-2020  rillig make(1): extract Var_DeleteVar from Var_Delete
 1.735  19-Dec-2020  rillig make(1): extract Var_Undef from ParseDirective
 1.734  13-Dec-2020  rillig make(1): replace %zu with %u in printf calls

This is needed to compile bmake with GCC 2.8.1 on SunOS 5.9.

To support ancient systems like this, the whole code of usr.bin/make is
supposed to use only ISO C90 features, except for filemon, which is not
used on these systems.
 1.733  13-Dec-2020  rillig make(1): add str_basename to reduce duplicate code

The function basename from POSIX has a few unfortunate properties, it is
allowed to return a pointer to static memory. This is too unreliable,
therefore this trivial own implementation.
 1.732  13-Dec-2020  rillig make(1): remove dead code from GetVarnamesToUnexport

Now that the parsing of the directives is unified and strict, there is
no need anymore for the dispatched functions to check for unknown
directives. These functions don't even get the information to decide
that since this decision is already done.
 1.731  13-Dec-2020  rillig make(1): clean up Var_Export
 1.730  13-Dec-2020  rillig make(1): replace bitset VarExportFlags with enum VarExportMode

The previous flags were not combined at all.
 1.729  12-Dec-2020  rillig make(1): eliminate boolean argument of Var_Export
 1.728  12-Dec-2020  rillig make(1): extract ExportVarsExpand from Var_Export
 1.727  12-Dec-2020  rillig make(1): fix undefined behavior when exporting ${:U }
 1.726  12-Dec-2020  rillig make(1): extract ExportVars from Var_Export
 1.725  12-Dec-2020  rillig make(1): rename Var_ExportVars to Var_ReexportVars
 1.724  12-Dec-2020  rillig make(1): error out on misspelled .export directives
 1.723  12-Dec-2020  rillig make(1): error out on misspelled .unexport-env
 1.722  12-Dec-2020  rillig make(1): inline local variable in ApplyModifiersIndirect
 1.721  12-Dec-2020  rillig make(1): in ApplyModifiersIndirect, rename local variable

In other places, the parsing position is abbreviated as pp as well.
 1.720  12-Dec-2020  rillig make(1): remove const from function parameters

These have been left-overs from refactoring, when these pieces were
extracted to separate functions.
 1.719  07-Dec-2020  rillig make(1): rename Var_Export1 to ExportVar

Since Var_Export1 is neither exported by the module nor does it belong
to the Var type, the previous function name was misleading. The 1 in
the function name was not as expressive as possible. The new name
aligns nicely with UnexportVar, which is a very young name as well.
 1.718  06-Dec-2020  rillig make(1): remove comment decoration
 1.717  06-Dec-2020  rillig make(1): move CleanEnv to UnexportVars

It clearly didn't belong in the GetVarnamesToUnexport part of the code.
 1.716  06-Dec-2020  rillig make(1): extract UnexportVars from Var_UnExport
 1.715  06-Dec-2020  rillig make(1): clean up GetVarnamesToUnexport
 1.714  06-Dec-2020  rillig make(1): extract GetVarnamesToUnexport from Var_UnExport
 1.713  06-Dec-2020  rillig make(1): replace pointer comparisons with enum

Keeping track of what it means if varname.str == str is not as
expressive as declaring what exactly to unexport.
 1.712  06-Dec-2020  rillig make(1): use FStr in Var_UnExport
 1.711  06-Dec-2020  rillig make(1): add FStr to var.c to make memory handling simpler

The pattern of having a variable and a corresponding freeIt variable
appears over and over again in make, so cast it into a struct.
 1.710  06-Dec-2020  rillig make(1): move type definitions in var.c to the top
 1.709  06-Dec-2020  rillig make(1): extract FreeEnvVar from Var_Parse
 1.708  06-Dec-2020  rillig make(1): inline macros for debug logging

No changes to the resulting binary, except for the line numbers in
assertions.
 1.707  05-Dec-2020  rillig make(1): define constants for enum zero-values
 1.706  05-Dec-2020  rillig make(1): remove redundant assignments
 1.705  05-Dec-2020  rillig make(1): extract ShuffleStrings from ApplyModifier_Order
 1.704  05-Dec-2020  rillig make(1): indent remaining code of var.c with tabs instead of spaces
 1.703  05-Dec-2020  rillig make(1): indent large parts of var.c with tabs instead of spaces

The few remaining functions need to be cleaned up before being indented
further, to reduce the overall indentation.
 1.702  05-Dec-2020  rillig make(1): improve explanation of tests for the SysV modifier
 1.701  04-Dec-2020  rillig make(1): extract UnexportEnv from Var_UnExport
 1.700  04-Dec-2020  rillig make(1): extract UnexportVar from Var_UnExport
 1.699  28-Nov-2020  rillig make(1): improve type of local variable in Var_Export1
 1.698  28-Nov-2020  rillig make(1): rename local variable in ModifyWord_Root

The name 'dot' conflicted with the global variable from dir.c when make
was compiled in all-in-one mode.
 1.697  28-Nov-2020  rillig make(1): rename local variable in ModifyWord_Suffix

It conflicted with 'dot' from dir.c.
 1.696  24-Nov-2020  rillig make(1): rename local variable in ApplyModifier_Assign
 1.695  23-Nov-2020  rillig make(1): use comparisons in boolean expressions

The generated code stays exactly the same.
 1.694  23-Nov-2020  rillig make(1): use properly typed comparisons in boolean contexts
 1.693  21-Nov-2020  rillig make(1): rename GNode.context to vars

Especially in the variables module, the expression 'ctxt->context' looked
redundant. Having a GNode as a context sounds ok, but a context of a
context just doesn't make sense.
 1.692  21-Nov-2020  rillig make(1): clean up freeing of environment variables in Var_Parse

The previous code with the extra boolean variable was a brain-twister
since the responsibility of freeing the memory was distributed over 3
different functions.
 1.691  21-Nov-2020  rillig make(1): clean up parameter order in EvalUndefined
 1.690  21-Nov-2020  rillig make(1): rename local variable in Var_Subst
 1.689  17-Nov-2020  rillig make(1): fix error message of failed :!cmd! modifier (since 2000-04-29)
 1.688  16-Nov-2020  rillig make(1): rename Targ_NewGN to GNode_New

This function is a classical constructor function, and if it weren't for
CLEANUP mode, it would have no dependencies on anything else besides the
memory allocator. Therefore it doesn't really matter which module
defines this function, and there is no need for the "Targ" to be part of
the function name.
 1.687  15-Nov-2020  rillig make(1): add remarks to var.c and the test varmod-match
 1.686  15-Nov-2020  rillig make(1): rename ApplyModifiersState.v to var
 1.685  14-Nov-2020  rillig make(1): replace a few HashTable_CreateEntry with HashTable_Set

Instead of HashTable_CreateEntry and HashEntry_Set, several places just
need the HashEntry for storing a value in it. This makes the calling
code simpler to understand.

These parts of the code are already hard enough to understand since they
are about memory management and aliasing. Having a too detailed API for
the HashTable only distracts from these topics.
 1.684  10-Nov-2020  rillig make(1): use consistent definition for MAKE_INLINE
 1.683  08-Nov-2020  rillig make(1): move lint option away from the debug flags

The lint option affects much more than a typical debug flag. Not only
does it produce additional diagnostics, it also changes the control flow
and fixes the traditional error handling in a few places.

No functional change.
 1.682  08-Nov-2020  rillig make(1): clean up code related to VarEvalFlags

Mention VARE_WANTRES before VARE_UNDEFERR since the latter depends on
the former.

In ApplyModifier_Assign, VARE_KEEP_DOLLAR doesn't have to be removed
from eflags since ParseModifierPart does this already.

In EvalUndefined, testing for VARE_WANTRES is redundant if VARE_UNDEFERR
is already set.
 1.681  08-Nov-2020  rillig make(1): clean up VarEvalFlags in the calls to Var_Parse and Var_Subst

There are only 3 flags, and some combinations don't even make sense.

VARE_UNDEFERR only makes sense if combined with VARE_WANTRES. If the
latter is not set, the variable expressions are only parsed, without
asking whether they are defined or not. Therefore, VARE_UNDEFERR cannot
have any effect in that case.

VARE_KEEP_DOLLAR is actively ignored by ParseModifierPart. In cases
where VARE_WANTRES is not set, this means that VARE_NONE can be passed,
which is easier to grasp than any bitset operations. This also gets rid
of a few type casts from enum to unsigned int that were necessary to
pass WARNS=6.
 1.680  08-Nov-2020  rillig make(1): rename local variable in VarSubstNested
 1.679  08-Nov-2020  rillig make(1): reorder branches in Var_Subst

This way, '$$' is handled first, followed by '$', followed by everything
else. This is easier to follow than first '$$' then not '$' and finally
'$'.
 1.678  08-Nov-2020  rillig make(1): extract VarSubstNested from Var_Subst

Each of these functions now fits on a moderately large screen.
 1.677  08-Nov-2020  rillig make(1): rename VARE_ASSIGN to VARE_KEEP_DOLLAR

The other flags in VarEvalFlags already describe their effects, not the
place where they are used. It's more important to know the effect.

Only a single unit test had to be adjusted. This probably means that
there are too few tests where the special effects of VARE_KEEP_DOLLAR
come into play. It could also mean that the effects are so simple and
obvious that they don't need any debug log, but that's not the case.
 1.676  08-Nov-2020  rillig make(1): use strict typing in conditions of the form !var
 1.675  07-Nov-2020  rillig make(1): fix type mismatch between int and VarExportFlags
 1.674  07-Nov-2020  rillig make(1): fix type mismatch between int and VarSetFlags
 1.673  07-Nov-2020  rillig make(1): make API of Buf_Init simpler

In most cases, the caller doesn't want to specify the exact number of
preallocated bytes.
 1.672  07-Nov-2020  rillig make(1): clean up code stylistically

* Replace character literal 0 with '\0'.
* Replace pointer literal 0 with NULL.
* Remove redundant parentheses.
* Parentheses in multi-line conditions are not redundant at the
beginning of a line.
* Replace a few !ptr with ptr == NULL.
* Replace a few ptr with ptr != NULL.
* Replace (expr & mask) == 0 with !(expr & mask).
* Remove redundant braces for blocks in cases where the generated code
stays the same. (Assertions further down in the code would get
different line numbers.)
* Rename parameters in CondParser_String to reflect the data flow.
* Replace #ifdef notdef with #if 0.

The generated code stays exactly the same, at least with GCC 5.5.0 on
NetBSD 8.0 amd64 using the default configuration.
 1.671  07-Nov-2020  rillig make(1): rename Var_Set_with_flags to Var_SetWithFlags

Now that the function is exported from the var module, it should stick
to the naming conventions for public functions.
 1.670  07-Nov-2020  rillig make(1): rename VarSet_Flags to VarSetFlags

Most other types don't have an underscore as well.
 1.669  06-Nov-2020  rillig make(1): rename getBoolean and s2Boolean
 1.668  06-Nov-2020  rillig make(1): rename local variable in Var_Parse
 1.667  06-Nov-2020  rillig make(1): format source code of a few functions in var.c
 1.666  05-Nov-2020  rillig make(1): update and clean up documentation of Var_Parse
 1.665  05-Nov-2020  rillig make(1): fix parameter name in ApplyModifiersIndirect and ApplyModifiers

The variable must be properly initialized before these functions are
called.
 1.664  05-Nov-2020  rillig make(1): clean up ParseVarnameLong

A parameter named pp is usually used as the parsing position, which is
updated upon successful return. Not so in ParseVarnameLong, where it
was updated in the unsuccessful branch only.

To avoid confusion, rename it to out_FALSE_pp, which is a longer name
but expresses more clearly what actually happens.
 1.663  05-Nov-2020  rillig make(1): extract EvalUndefined from ParseVarnameLong
 1.662  05-Nov-2020  rillig make(1): extract FindLocalLegacyVar from Var_Parse
 1.661  05-Nov-2020  rillig make(1): replace '(' and ')' with neutral '\0' in Var_Parse

The only possible values for extramodifiers are "H:" and "T:", therefore
parsing is independent of startc and endc. Use '\0' instead of '(' and
')' to remove any possible confusion about how '{' and '}' would be
handled.
 1.660  05-Nov-2020  rillig make(1): clean up documentation for Var_Subst
 1.659  05-Nov-2020  rillig make(1): remove redundant parentheses from sizeof operator

The parentheses are only needed if the argument is a type, not an
expression.
 1.658  05-Nov-2020  rillig make(1): remove wrong comment from Var_Subst

The error handling in variable expressions is inconsistent. Some errors
are detected, most aren't. In particular, the error message for
undefined variables is _not_ issued on undefined variables but instead
on parse errors.
 1.657  04-Nov-2020  rillig make(1): negate discardUndefined to preserveUndefined
 1.656  04-Nov-2020  rillig make(1): add test for undefined variables in command line arguments

The variable discardUndefined has an implicit negation in its name,
which makes it hard to understand. Plus, most of the time it is true.
It's better to have a flag that is false most of the time and has a
positive name.

On the first attempt of inverting that variable, I stumbled upon
MainParseArgs, which initially leaves discardUndefined == FALSE, and
after handling the dashed options, sets it to TRUE. This would make a
difference when more command line arguments would be added later via the
.MAKEFLAGS special target.

Upon further inspection, the only place where discardUndefined is used
is in VarAssign_EvalSubst in parse.c, and that place is not reachable
from any of the dashed options. Therefore, discardUndefined could
already be set at the very beginning of MainParseArgs or even when
initializing the global variable itself, without any observable
difference.

Not even the ::= variable modifier could do anything about this since it
is not reachable from the dashed command line options as well, and in
addition, it expands its right-hand side in any case, always discarding
undefined variables. Oh, these little inconsistencies everywhere.
 1.655  04-Nov-2020  rillig make(1): rename oldVars to discardUndefined

While here, moved all the documentation about this variable into a
single place.
 1.654  04-Nov-2020  rillig make(1): remove redundant condition from ApplyModifiersIndirect

Whenever varUndefined is returned from another function, that is only
done if eflags does not contain VARE_UNDEFERR. Therefore, testing for
that flag is unnecessary.
 1.653  04-Nov-2020  rillig make(1): replace emptyString with allocated empty string

Special-casing this variable only made the code more complicated.
Furthermore, it is not related to error handling in any way and
therefore distracted the reader from this topic.
 1.652  02-Nov-2020  rillig make(1): fix error handling on parse errors in variable expressions

This change doesn't change any of the unit tests since the error
handling code is not yet complete, see the many "handle errors" in the
code. Nevertheless, the "out_FALSE_res = VPR_PARSE_MSG" was wrong since
the error message was only printed in lint mode, not in default mode.
 1.651  02-Nov-2020  rillig make(1): improve local variable name in ParseVarname
 1.650  02-Nov-2020  rillig make(1): document that skipping a modifier on parse errors is risky
 1.649  02-Nov-2020  rillig make(1): error out on unclosed expressions after the colon
 1.648  02-Nov-2020  rillig make(1): remove word "Ptr" from variable names

Whether or not a variable is a pointer is obvious from the context.
Since the introduction of function prototypes in C90, this information
is checked by the compiler and no longer needs to be encoded in the
variable names.
 1.647  02-Nov-2020  rillig make(1): fix wording of a comment in var.c

The "why again" could be easily misunderstood, it was ambiguous.
 1.646  02-Nov-2020  rillig make(1): add a comment where to fix the STOP/STORE test from varmod.mk
 1.645  02-Nov-2020  rillig make(1): format code in ApplyModifiersIndirect and ApplyModifiers
 1.644  02-Nov-2020  rillig make(1): merge variables p and nested_p in ApplyModifiersIndirect

When the code was still in ApplyModifiers, the variable nested_p was
necessary to distinguish the parsing position in the nested modifier
from the parsing position of the main expression.
 1.643  02-Nov-2020  rillig make(1): rename rval to mods in ApplyModifiersIndirect
 1.642  02-Nov-2020  rillig make(1): inline and rename variables in ApplyModifiersIndirect
 1.641  01-Nov-2020  rillig make(1): extract ApplyModifiersIndirect from ApplyModifiers
 1.640  01-Nov-2020  rillig make(1): rename local variable in ApplyModifiers
 1.639  01-Nov-2020  rillig make(1): rename local variables in ApplyModifier_SysV
 1.638  01-Nov-2020  rillig make(1): reduce number of nested parentheses in ApplyModifier_Assign
 1.637  01-Nov-2020  rillig make(1): revert unintended change from previous commit
 1.636  01-Nov-2020  rillig make(1): fix indentation of source code
 1.635  01-Nov-2020  rillig make(1): treat malformed :range, :ts and :[...] as errors

Before, integer overflow in the :[1..2] modifier had not been detected,
and the actual behavior varied between ILP64 and LP64I32 machines.

Before, the :ts modifier accepted character literals like \012345 and
\x1F600, which don't fit in a single character and were thus truncated.

Before, the :range modifier issued an "Unknown modifier" error message
for :range=x, which was not quite correct. The error message in this
case is now "Invalid number".
 1.634  01-Nov-2020  rillig make(1): add tests for the variable modifiers :[words] and :range
 1.633  01-Nov-2020  rillig make(1): clean up comments in ApplyModifier_Words
 1.632  31-Oct-2020  rillig make(1): rename parameter of Var_Value
 1.631  31-Oct-2020  rillig make(1): make parsing of the :gmtime and :localtime modifiers stricter

These variable modifiers accept an optional timestamp in seconds, to
select which date to print. This feature is only used very rarely. The
NetBSD build doesn't use it at all, and the FreeBSD build mainly uses
the plain modifiers :gmtime and :localtime, but not their optional
argument :gmtime=1500000000.

Therefore, this change is not going to affect many builds. Those that
are indeed affected had been wrong all the time anyway.

At parse time, these errors stop the build, as intended. After that,
when the actual shell commands of the targets are expanded and run,
these errors don't stop anything, the build just continues as if nothing
had happened. This is a general problem with Var_Parse, see the many
"handle errors" markers in the code. Another problem is that on parse
errors, parsing continues and spits out spurious strings of the form
"mtime" and "ocaltime". This as well is a general problem with error
handling in make.

ok sjg
 1.630  31-Oct-2020  rillig make(1): format #include directives consistently
 1.629  31-Oct-2020  rillig make(1): rename ismeta to is_shell_metachar

The old name was too broad.
 1.628  31-Oct-2020  rillig make(1): remove debug logging for the :Q variable modifier

The same information is already logged in LogAfterApply.
 1.627  31-Oct-2020  rillig make(1): fix local variable names in ParseModifierPart
 1.626  31-Oct-2020  rillig make(1): rewrite VarIsDynamic to VarnameIsDynamic

The condition for the context is the same for both short and long names,
therefore move that condition to the only caller.

Clean up the comment and move its parts to the appropriate places. The
"with the dollar sign escaped" part had been wrong already in 1993, and
it didn't get better over time.
 1.625  31-Oct-2020  rillig make(1): rename local variable freePtr in Var_Parse
 1.624  31-Oct-2020  rillig make(1): use consistent parameter order in varname parsing functions
 1.623  31-Oct-2020  rillig make(1): extract ParseVarnameLong from Var_Parse
 1.622  31-Oct-2020  rillig make(1): extract ParseVarnameShort from Var_Parse

With its more than 200 lines, the latter has too much code to be read
easily.
 1.621  31-Oct-2020  rillig make(1): fix type of ParseModifierPart parameter delim
 1.620  31-Oct-2020  rillig make(1): extract IsEscapedModifierPart from ParseModifierPart
 1.619  31-Oct-2020  rillig make(1): clean up ModifyWords

Reorder the parameters to match the documentation comment, and the
remaining parameters in chronological order. Remove the unused
parameter ctxt. The callbacks that need it pass it in their
modifyWordArgs instead.
 1.618  31-Oct-2020  rillig make(1): add test for combining the :@ and :? variable modifiers
 1.617  31-Oct-2020  rillig make(1): remove redundant condition for regmatch_t.rm_eo being -1

If rm_so is -1, rm_eo is guaranteed to be -1 as well.
 1.616  31-Oct-2020  rillig make(1): do not look up local variables like .TARGET anywhere else

Nobody defines a global variable named .TARGET since that would have
many unpredictable effects, applying to all targets at once.

Nobody defines an environment variable named .TARGET since that's
against the naming conventions for environment variables and would have
the same effect.

Because of this, there is no point looking up the variables that are
local to a GNode anywhere else. This means they cannot come from the
environment and thus their value doesn't need to be freed after use,
which makes the code simpler.

The newly added accessor functions in make.h refer to external
functions, but since that header is not used anywhere outside of
usr.bin/make, it doesn't matter. Between 2020-08-25 and 2020-10-30,
that header had been referenced by usr.bin/xinstall.
 1.615  31-Oct-2020  rillig make(1): reduce the scope where recursive expressions are detected

Only the call to Var_Subst needs to be protected since the other
functions have nothing to do with expanding variables.
 1.614  31-Oct-2020  rillig make(1): fix indentation in source code
 1.613  31-Oct-2020  rillig make(1): remove redundant null check from s2Boolean
 1.612  31-Oct-2020  rillig make(1): remove redundant code branch in SysVMatch

The general code path of parsing :from=to achieves the same effect.
 1.611  31-Oct-2020  rillig make(1): merge duplicate code in ModifyWord_Subst
 1.610  30-Oct-2020  rillig make(1): rename Str_SYSVMatch and its parameters
 1.609  30-Oct-2020  rillig make(1): improve variable name in Var_Value, document callback
 1.608  30-Oct-2020  rillig make(1): remove unsatisfiable conditions in Var_Set_with_flags
 1.607  30-Oct-2020  rillig make(1): document where the variable name is expanded
 1.606  30-Oct-2020  rillig make(1): change char * to void * in Var_Value

The only purpose of the parameter freeIt is to free the memory
associated with the return value. To do this, no pointer arithmetic is
needed. Therefore, change to a void pointer, to catch accidental use of
that pointer.
 1.605  30-Oct-2020  rillig make(1): clean up comments and local variables in var.c
 1.604  30-Oct-2020  rillig make(1): fix documentation for VARP_SUB_ONE

The pattern is only replaced a single time, not everywhere in the first
matching word.
 1.603  30-Oct-2020  rillig make(1): make iterating over HashTable simpler
 1.602  30-Oct-2020  rillig make(1): rename VAR_EXPORTED_YES to VAR_EXPORTED_SOME

The "yes" sounded too much like "all".
 1.601  30-Oct-2020  rillig make(1): document VAR_READONLY, now that it is really read-only
 1.600  30-Oct-2020  rillig make(1): prevent appending to read-only variables
 1.599  30-Oct-2020  rillig make(1): fix indentation in source code
 1.598  30-Oct-2020  rillig make(1): extract MayExport from Var_Export1
 1.597  30-Oct-2020  rillig make(1): remove redundant evaluations in VarFind
 1.596  30-Oct-2020  rillig make(1): replace VarFindFlags with a simple Boolean

Either all flags had been given or none. Except in Var_Append, but
since the ctxt was VAR_GLOBAL anyway, adding FIND_GLOBAL there did not
make a difference.
 1.595  30-Oct-2020  rillig make(1): rename FIND_CMD to FIND_CMDLINE, to match VAR_CMDLINE

While here, use a consistent order for the enum constants. This is both
the declaration order and at the same time the usual lookup order,
unless the -e option is given.
 1.594  30-Oct-2020  rillig make(1): rename VAR_CMD to VAR_CMDLINE

Since make has to do with both the command line and child commands, the
former name was confusing.
 1.593  30-Oct-2020  rillig make(1): rename SAVE_DOLLARS to follow the naming conventions
 1.592  30-Oct-2020  rillig make(1): clean up and update module comment in var.c
 1.591  27-Oct-2020  rillig make(1): initialize all CmdOpts fiels
 1.590  26-Oct-2020  rillig make(1): group the command line options and arguments

By having a single struct that holds all command line options and
arguments, it is easy to see in the code when such a command line
argument is modified. It also cleans up the namespace since the command
line options don't follow a common naming style. Having them in a
struct also means that there is a single place for putting the
documentation, not two as before.

The struct also suggests to extract the initialization code out of main,
which is still too large, having more than 400 lines of code and
covering far too many topics.
 1.589  25-Oct-2020  rillig make(1): add GNode_Path to access the path of a GNode
 1.588  25-Oct-2020  rillig make(1): rename hash functions to identify the type name

This makes it easier to spot mismatches between the function name and
its first parameter, although the compiler should already catch most of
them. Except for void pointers.
 1.587  25-Oct-2020  rillig make(1): reduce duplicate code in VarFind
 1.586  25-Oct-2020  rillig make(1): reduce amount of string hashing

In pkgsrc, running "bmake show-all" in pkgtools/pkglint called the hash
function 249130 times before, and only 115502 times after.

Still, a single call to Var_Set hashes the same string 3 times.
 1.585  25-Oct-2020  rillig make(1): replace PtrVector with Vector, which can contain any type
 1.584  25-Oct-2020  rillig make(1): rename type Vector to PtrVector

This allows the name Vector to be used for a more generic vector type,
which will be added soon.
 1.583  24-Oct-2020  rillig make(1): remove macros MIN and MAX

These macros typically evaluate one of their arguments twice. Until
2020-08-31, they had not parenthesized their arguments properly. They
are only used in a few places, therefore it doesn't hurt much to have
them expanded.
 1.582  23-Oct-2020  rillig make(1): move comment in Var_Set_with_flags

Until 2000-05-11, that comment was just above a VarFind call. 20 years
later, it is back again where it belongs.
 1.581  22-Oct-2020  rillig make(1): remove redundant type casts

This mainly affects the void pointers in callback functions for lists.
These had been necessary once when the parameter type was still
ClientData instead of void pointer.
 1.580  22-Oct-2020  rillig make(1): extract CanonicalVarname from VarFind
 1.579  20-Oct-2020  rillig make(1): document parameter to Var_Parse
 1.578  18-Oct-2020  rillig make(1): add tags to enum types

This allows IDEs to offer better type information than "anonymous enum".
 1.577  18-Oct-2020  rillig make(1): rename HashEntry.name to key
 1.576  18-Oct-2020  rillig make(1): remove underscore from Hash_Table and Hash_Entry

For consistency with the other type names, such as GNodeListNode.
 1.575  18-Oct-2020  rillig make(1): make API for iterating over hash tables simpler
 1.574  18-Oct-2020  rillig make(1): rename Stack to Vector

Both Var_Dump and GetActuallyIncludingFile access more than only the top
item of the stack, therefore it is more honest to rename the data type.
 1.573  18-Oct-2020  rillig make(1): sort variables in debug output

This way it's easier to see whether an expected variable is missing, or
to compare the values of related variables, since they usually share a
common prefix. Any sorting criterion is better than the hash code.

Sorting the variables by name changed the order of the output in
varname.mk. That test didn't test anything meaningful, it was just a
playground to understand and demonstrate the current implementation of
how the variables are stored, therefore it has been removed.
 1.572  17-Oct-2020  rillig make(1): normalize initialization and cleanup of the modules
 1.571  17-Oct-2020  rillig make(1): fix indentation
 1.570  06-Oct-2020  rillig make(1): rework memory allocation for the name of variables

There's more to know about variable names than fits in a one-liner.
While here, enforce that the name is not modified by splitting it into
the established (var + var_freeIt) pattern.

Since the name is not modified and not freed in the middle of evaluating
an expression, there is no need to make a backup copy of it. That code
had been necessary more than 12 years ago, but not anymore since the
code got a lot cleaner since then.
 1.569  06-Oct-2020  rillig make(1): remove duplicate code for creating variables
 1.568  05-Oct-2020  rillig make(1): fix compilation with GCC 10 and -Wimplicit-fallthrough=4
 1.567  05-Oct-2020  rillig make(1): revert previous commit

It had accidentally reverted all the work from the past few days.
 1.566  05-Oct-2020  rillig make(1): fix double-free bug in -DCLEANUP mode (since 2020-10-02)

The bug had been introduced with dir.c 1.155 on 2020-10-02 22:20:25. In
that commit, openDirectories was replaced with a combination of a list
with a hash table, for more efficient lookup by name.

Upon cleanup, OpenDirs_Done is called, which in turn called
Dir_ClearPath. Dir_ClearPath takes full ownership of the given list and
empties it. This was no problem before since afterwards the list was
empty and calling Lst_Free just frees the remaining list pointer.

With OpenDirs, this list was combined with a hash table, and the hash
table contains the list nodes, assuming that the OpenDirs functions have
full ownership of both the list and the hash table. This assumption was
generally correct, except for the one moment during cleanup where full
ownership of the list was passed to Dir_ClearPath, while the hash table
still contained pointers to the (now freed) list nodes. This by itself
was not a problem since the hash table would be freed afterwards. But
as part of Dir_ClearPath, OpenDirs_Remove was called, which looked up
the freed directory by name and now found the freed list node, trying to
free it again. Boom.

Fixed by replacing the call to Dir_ClearPath with code that only frees
the directories, without giving up control over the list.
 1.565  04-Oct-2020  rillig make(1): only use the VARE_ASSIGN flag if necessary

When checking the right-hand side of a variable assignment for syntax
errors, it does not matter whether a '$$' is expanded to '$' or kept as
'$$'.
 1.564  03-Oct-2020  rillig make(1): use consistent pattern for parsing whitespace

The pp and cpp in the function names stand for "parsing position" and
"const parsing position".
 1.563  03-Oct-2020  rillig make(1): inline Str_FindSubstring in ModifyWord_Subst
 1.562  03-Oct-2020  rillig make(1): extract ApplyModifier_Literal into separate function
 1.561  03-Oct-2020  rillig make(1): extract ApplyModifier_Unique into separate function
 1.560  03-Oct-2020  rillig make(1): in lint mode, reject modifiers without delimiter

The expression ${VAR:LPL} must now be written as ${VAR:L:P:L}. The
manual page has never documented that some modifiers don't need to be
delimited by ':' and others need to. That would have been unnecessarily
confusing anyway.
 1.559  03-Oct-2020  rillig make(1): ignore -env and -literal in assignments to .MAKE.EXPORTED

Previously, assigning the string "-env" to the variable .MAKE.EXPORTED
had the same effect as the .export-env directive. This was only due to
a sloppy implementation, not by design.

For the string "-literal" and the directive .export-literal, the
situation was even worse since the actually executed code was a wild
mixture between .export and .export-literal that in the end exported the
expanded form of the variable. Therefore there was no practical use
case of this implementation flaw.
 1.558  03-Oct-2020  rillig make(1): do not export variable names starting with '-'

By convention, names of environment variables consist of uppercase
letters and underscores. Most of them start with an uppercase letter.
In a few cases, the names also contain lowercase letters, such as in
http_proxy.

Variable names starting with a hyphen are confusing and might be
mistaken as command line options. Therefore don't export these.

This also affects a few edge cases that don't occur in practice, such as
setting .MAKE.EXPORTED=-env or .MAKE.EXPORTED=-literal. These had not
worked as expected anyway.
 1.557  03-Oct-2020  rillig make(1): clean up confusing code in Var_Export

The generated code stays exactly the same.
 1.556  02-Oct-2020  rillig make(1): in ApplyModifier_To, update pp in each branch

Before, the parsing position was updated once at the beginning, which
didn't make sense. Updating it in each branch allows to decide for its
appropriate value in each branch individually.
 1.555  30-Sep-2020  rillig make(1): extract ApplyModifier_Quote into separate function
 1.554  30-Sep-2020  rillig make(1): fix bug in evaluation of indirect variable modifiers

Since 2020-09-22, when indirect variable modifiers were applied to a
variable expression based on an undefined variable, these modifiers did
not change the state of a variable expression from undefined to defined.
The modifiers that do this are :D, :U, :L, :P. Minimal example:

${VARNAME:${:UL}}

The :L modifier from the inner expression sets the value of the outer
expression to its variable name, in this case "VARNAME". The outer
expression was not marked as being defined though, which resulted in a
"Malformed conditional" error.

In the commit from 2020-09-22, vardebug.exp had changed a lot, and I had
not inspected the change closely. The important detail was in lines 56
and 60, where VAR_JUNK|VAR_KEEP changed into VEF_UNDEF, thereby losing
the VEF_DEF bit.
 1.553  29-Sep-2020  rillig make(1): rename ShortVarValue to UndefinedShortVarValue
 1.552  29-Sep-2020  rillig make(1): remove unused assignments to local variables

These only became obvious once ApplyModifier had been extracted, which
shrank the source code of ApplyModifiers by about 100 lines.

Even GCC 5 had seen that they are redundant and had not generated any
code for them.
 1.551  29-Sep-2020  rillig make(1): extract ApplyModifier from ApplyModifiers

This allows a return, instead of a variable assignment followed by a
break, saving a few lines of source code.

The size of the resulting binary shrinks as well, even though the code
is essentially the same as before.
 1.550  28-Sep-2020  rillig make(1): make debug logging simpler

This avoids referring to the debug_file variable in many places where
this implementation detail is not necessary.
 1.549  28-Sep-2020  rillig make(1): extract logging from ApplyModifiers into separate functions

These two logging blocks are massive enough to disturb the reading flow
of the remaining code.

Even without these two blocks, ApplyModifiers is still 250 lines long,
which is quite much.
 1.548  28-Sep-2020  rillig make(1): move variable modifier :sh into separate function
 1.547  28-Sep-2020  rillig make(1): make debug logging in var.c compatible to C90

The ... ellipsis for macros is only available since C99.
 1.546  27-Sep-2020  rillig make(1): normalize whitespace in source code

There is no more space tab. Either only tabs or only spaces or tabs
followed by spaces, but not spaces followed by tabs.
 1.545  27-Sep-2020  rillig make(1): rename Buf_Size to Buf_Len

The new name better matches the field name Buffer.len as well as the
variables around the calls to this function.
 1.544  27-Sep-2020  rillig make(1): replace direct access to Buffer fields with inline function

This way, renaming the fields of the buffer is restricted to only buf.h
and buf.c.
 1.543  26-Sep-2020  rillig make(1): clean up API for finding and creating GNodes

The previous API had complicated rules for the cases in which the single
function returned NULL or what it did. The flags for that function were
confusing since passing TARG_NOHASH would create a new node even though
TARG_CREATE was not included in that bit mask.

Splitting the function into 3 separate functions avoids this confusion.
It also reveals several places where the complicated API led to
unreachable code. Such code has been removed.
 1.542  26-Sep-2020  rillig make(1): add Hash_FindValue, for direct access to hash table data
 1.541  25-Sep-2020  rillig make(1): add tags to some of the unnamed structs

The tags prevent the structs from accidentally becoming compatible
types.

While here, remove a few typedefs for structs that are single-purpose,
since there is no point in abstracting from the actual representation of
these types.
 1.540  25-Sep-2020  rillig make(1): fix type of local variable in ParseModifierPart

Thanks, Clang-Tidy.
 1.539  25-Sep-2020  rillig make(1): fix type of local variable in ApplyModifiers

Thanks, Clang-Tidy.
 1.538  25-Sep-2020  rillig make(1): document expansion of undefined variables in Var_Subst

From reading the code alone, it is not obvious what effects this
innocent-looking code has.
 1.537  25-Sep-2020  rillig make(1): rename local variables in Var_Subst

The variable name "str" did not make it clear enough that the pointer is
constantly moving, to parse the given string. The name "p" expresses
this more clearly.
 1.536  23-Sep-2020  rillig make(1): fix unexpected behavior in ::= variable modifier

Previously, the ::= modifier had returned an error value, which caused
the variable expression to be preserved. This behavior was not useful
in this case; it had only been intended to be used for undefined
variables.

To fix it, distinguish between parse errors, undefined variables and
regular empty strings.
 1.535  23-Sep-2020  rillig make(1): inline local variable delim

Now that ParseModifierPart generates the error message itself, there is
no need to keep this variable around since it is read a single time.
 1.534  22-Sep-2020  rillig make(1): condense code in Var_Subst
 1.533  22-Sep-2020  rillig make(1): prepare Var_Subst for proper error handling

Returning a VarParseResult instead of a string makes it possible to let
the error bubble up, until it reaches the main expression.
 1.532  22-Sep-2020  rillig make(1): remove obsolete fix for PR bin/29985
 1.531  22-Sep-2020  rillig make(1): prepare error handling in nested variable expressions

Having ParseModifierPart return VarParseResult allows to report the
errors from nested variable expressions.
 1.530  22-Sep-2020  rillig make(1): handle errors about missing delimiter in a single place
 1.529  22-Sep-2020  rillig make(1): move ParseModifierPart further down in the code

This way, it can access the ApplyModifierState, which will be used in a
follow-up commit to reduce the code duplication around the error
handling for missing delimiters.
 1.528  22-Sep-2020  rillig make(1): remove remaining references to VAR_JUNK and VAR_KEEP
 1.527  22-Sep-2020  rillig make(1): move VAR_JUNK and VAR_KEEP away from VarFlags

These two flags have nothing to do with a variable. They are only used
while evaluating a variable expression.

While here, rename the flags and make their documentation more precise.
 1.526  22-Sep-2020  rillig make(1): rename VarParseErrors to VarParseResult

The name VPE_OK was confusing since it was not an error at all.
 1.525  22-Sep-2020  rillig make(1): extract common code from the ApplyModifier functions
 1.524  22-Sep-2020  rillig make(1): use fine-grained type names for lists and their nodes

This is only intended to help the human reader. There is no additional
type safety yet.
 1.523  21-Sep-2020  rillig make(1): clean up VarParseErrors, for proper error handling

Having a pure bitset was wrong.

Instead, there are several alternatives (parse error, eval error, undef
error), and each of them can either have an error message printed (good)
or not (bad). In addition, there are VPE_OK for successful expression
evaluation and VPE_UNKNOWN (only used during migration to the correct
error handling scheme).
 1.522  14-Sep-2020  rillig make(1): fix wrong error for undefined variables in lint mode
 1.521  14-Sep-2020  rillig make(1): inline character constants in var.c

This removes a level of indirection, and the macro names were quite
similar in appearance as well. The macros may have been used by people
whose editors don't recognize C string and character literals when
navigating to the corresponding brace or parenthesis. These editors had
already been confused before this commit since there are also string
literals with unbalanced parentheses, and there are far fewer
compensating comments such as /*{*/ or /*)*/ in the code. In fact, the
only such comment that is left over was in for.c.

This way, there is now a single consistent way of writing these
character literals, which is without macros or comments, since that is
the simplest form.
 1.520  14-Sep-2020  rillig make(1): don't require nested variables to be defined in conditions

This code only applies to lint mode (-dL) for now. After a test phase
and a thorough check for possible side effects, it will be activated in
normal mode, too. Having this code in lint mode is required to run
src/build.sh, which relies a lot on using variables with undefined
nested variables in conditions.

In the default mode, any errors about nested undefined variables are not
printed since in Var_Subst, oldVars is true. Therefore, it is not urgent
to properly handle these nested variables correctly there.
 1.519  13-Sep-2020  rillig make(1): in Var_Parse, replace bmake_strldup with bmake_strsedup
 1.518  13-Sep-2020  rillig make(1): in lint mode, report undefined variables in conditions
 1.517  13-Sep-2020  rillig make(1): suppress wrong "Malformed conditional" for undefined variables

This only has an effect in lint mode right now.
 1.516  13-Sep-2020  rillig make(1): in lint mode, improve error handling for undefined variables

It's a first step for improving the error message that make prints.
 1.515  13-Sep-2020  rillig make(1): make documentation of VarParseErrors more precise
 1.514  13-Sep-2020  rillig make(1): prepare Var_Parse for proper error handling and reporting

Right now, Var_Parse swallows many errors during parsing and evaluation.
Ideally, these errors should propagate from the deeply nested
expressions where they occur up to the top-level expressions. When such
an error occurs, the depending expressions should not be evaluated any
further. They may still be parsed, but side effects should be
minimized.

The goal is to prevent incomplete expressions like the "xy}" in
moderrs.exp:106 from being evaluated and eventually passed to the shell
for execution. This expression is a left-over from a parse error in the
mod-t-parse target in moderrs.mk:154.

This commit is a first step in analyzing and verifying the current state
of affairs. The modelling in VarParseErrors already looks complicated
but is expected to closely match reality.
 1.513  13-Sep-2020  rillig make(1): shorten debug output of ApplyModifiers

Having the words "eflags" and "vflags" in the debug output was too
repetitive. That they are flags is made obvious by the '|' separator,
and the flags have clearly distinguishable names (VARE_* vs. VAR_*),
which lowers the chance for confusion.
 1.512  13-Sep-2020  rillig make(1): clean up RCSID blocks

These blocks mostly consisted of redundant structure, following the same
#ifndef pattern over and over, with only minimal variation.

It's easier to maintain if the common structure is only written once and
encapsulated in a macro.

To avoid "defined but unused" warnings from GCC in the case where
MAKE_NATIVE is not defined, I had to add volatile. Adding
MAKE_ATTR_UNUSED alone would not preserve the rcsid variable in the
resulting binary.
 1.511  13-Sep-2020  rillig make(1): clean up API for evaluating conditions

There was no need to make struct If publicly visible.

There was no need to have parameters in the public API that were passed
the same constants all the time.

The former function names had not been distinctive.
 1.510  13-Sep-2020  martin Initialize endc unconditionally, gcc complains and it is not obvious
whether this is a false positive.
 1.509  13-Sep-2020  rillig make(1): rename ApplyModifier_Exclam

The names of all other ApplyModifier functions already describe the
effect instead of the spelling.
 1.508  13-Sep-2020  rillig make(1): inline call to strchr in ValidShortVarname

It's a pity that neither GCC 5 nor GCC 10 nor Clang 9 inline this code
themselves, even though it would be easy to do.

Clang 9 at least replaces strchr with memchr, but that is still too
complicated for a simple "is this character one of these" question.

For a repeated "if (varname != ...)" instead of the switch, GCC 10
generates really boring and inefficient code, even though it is easy to
see that the order of the comparisons doesn't matter.
 1.507  13-Sep-2020  rillig make(1): in lint mode, complain about erroneous $$

Since 2008-12-21, make has silently ignored strange variable names in
constructs like '$$', '$}', '$' followed by nothing. Ignoring these
bugs in makefiles instead of reporting them is not a good idea.

To improve the situation, make complains about these errors now, but
only in lint mode (-dL). This preserves existing behavior while still
allowing to validate existing makefiles that they don't depend on this
bug.

If the test phase goes well, these error messages may be enabled
unconditionally.

https://mail-index.netbsd.org/pkgsrc-users/2020/09/12/msg032229.html
 1.506  13-Sep-2020  rillig make(1): improve implementation comment in Var_Parse
 1.505  12-Sep-2020  rillig make(1): fix return type of ApplyModifier_Localtime
 1.504  12-Sep-2020  rillig make(1): move buffer from Var_Parse to ParseVarname

There's no reason to keep the buffer in memory after the variable name
has been parsed.
 1.503  12-Sep-2020  rillig make(1): reword variable invocation to variable expression

Variables are a passive thing. They cannot be invoked, they can only be
evaluated.
 1.502  12-Sep-2020  rillig make(1): extract ShortVarValue from Var_Parse
 1.501  12-Sep-2020  rillig make(1): extract ParseVarname from Var_Parse

This was an easy part since it affects only a few variables. 250 lines
for a single function is still quite a lot, so further refactorings will
follow.
 1.500  12-Sep-2020  rillig make(1): rename local variable in Var_Subst
 1.499  12-Sep-2020  rillig make(1): remove double negation in Var_Parse
 1.498  12-Sep-2020  rillig make(1): clean up Var_Parse
 1.497  12-Sep-2020  rillig make(1): rename local variables in Var_Parse

The main property of the former "str" is not being a string but pointing
at the start of the expression to be parsed.

The main property of the former "tstr" is not being a string but being
the moving pointer, the current parsing position. No idea what the "t"
in "tstr" stood for.
 1.496  12-Sep-2020  rillig make(1): rename Var_ParsePP back to Var_Parse

The migration to the "parsing position" pointer has been done.
 1.495  12-Sep-2020  rillig make(1): migrate CondParser_String to Var_ParsePP
 1.494  12-Sep-2020  rillig make(1): fix inconsistent code indentation
 1.493  11-Sep-2020  rillig make(1): add wrappers around ctype.h functions

This avoids casting the argument to unsigned char, and to cast the
result of toupper/tolower back to char.
 1.492  11-Sep-2020  rillig make(1): replace *a->b with a->b[0]

This allows the code to be read strictly from left to right. In most
places this style was already used.
 1.491  08-Sep-2020  rillig make(1): fix off-by-one error in SuffExpandChildren

In suff.c r1.144 from yesterday, in the line "cp += nested_p - cp", I
accidentally removed the "- 1". Since these "- 1" lines lead to slow
execution, each branch now increments the pointer separately by the
actually needed amount.

Fixing this bug posed way more new questions than it answered, and it
revealed an inconsistency in the parser about how characters are to be
escaped, and missing details in the documentation of Var_Parse, as well
as a parse error that unexpectedly doesn't stop make from continuing.
 1.490  07-Sep-2020  rillig make(1): migrate to Var_ParsePP in Var_Parse and Var_Subst
 1.489  07-Sep-2020  rillig make(1): migrate ApplyModifiers to Var_ParsePP
 1.488  07-Sep-2020  rillig make(1): remove redundant includes
 1.487  04-Sep-2020  rillig make(1): add more explanation for undefined variable expressions
 1.486  03-Sep-2020  rillig make(1): migrate ApplyModifier_Defined to Var_ParsePP
 1.485  03-Sep-2020  rillig make(1): migrate Var_Parse API to parsing position

The ApplyModifier functions already use this pattern. For simplicity
and consistency Var_Parse should do the same. This saves a parameter to
be passed.

The migration takes place step by step, just like for the Lst functions
a few days ago.
 1.484  02-Sep-2020  rillig make(1): fix documentation of Var_Subst

The "var" parameter does not exist anymore.
 1.483  02-Sep-2020  rillig make(1): clean up comments in var.c, make VarQuote const-correct
 1.482  31-Aug-2020  rillig make(1): inline a local variable in VarUniq

Just to eliminate any remote possibility of aliasing and thereby
forgetting to update all involved variables.
 1.481  31-Aug-2020  rillig make(1): add more examples to the documentation of ParseModifierPart
 1.480  31-Aug-2020  rillig make(1): fix the :u modifier, which was broken for almost a day

Big thanks go to sjg, who discovered the bug and did the main work to
track it down.

In the unit tests for the :u modifier from the previous commit, I had
forgotten to actually add the :u modifier at the end. I added it now
and also added a few other tests. It's better to have a few more tests
than too few.
 1.479  30-Aug-2020  rillig make(1): replace brk_string with Str_Words

The API is much simpler, and there is less detail that is exposed by
default and fewer punctuation to type on the caller's side. To see that
there is some memory to be freed, one would have to look into the
struct. Having part of the return value as the actual return value and
the rest in output parameters was unnecessarily asymmetrical.
 1.478  29-Aug-2020  rillig make(1): trust that Var_Parse never returns NULL

That function is quite long, but all its return paths lead either to the
expanded variable expression, or to var_Error or varNoError.
 1.477  29-Aug-2020  rillig make(1): trust that Var_Subst never returns NULL

It really never does, and it doesn't even report errors. It just
returns the content of the buffer, up to the first parse error.
 1.476  29-Aug-2020  rillig make(1): add comments for ModifyWord_Subst

Without any comments, the code is unnecessarily hard to follow.
 1.475  29-Aug-2020  rillig make(1): split ApplyModifier_To into paragraphs
 1.474  29-Aug-2020  rillig make(1): add bmake_strsedup for duplicating a substring
 1.473  29-Aug-2020  rillig make(1): allow for strict type checking for Boolean

Having Boolean aliased to int creates ambiguities since int is widely
used. Allow to occasionally compile make with -DUSE_DOUBLE_BOOLEAN to
check that the type definitions still agree.
 1.472  25-Aug-2020  rillig make(1): fix error message for ::!= modifier with shell error

It's just too easy to confuse st->val and val.
 1.471  25-Aug-2020  rillig make(1): distinguish enum flags and values in debugging mode

When printing an enum value in debugging mode, distinguish between
bitsets containing flags and ordinary enums that just contain different
values.

Make the macros in enum.h more uniform. Provide a simple scheme for
defining the run-time type information of enums whose number of values
is a number with more than 2 bits set in the binary representation.
This case was not obvious before, and it was pure luck that the current
interesting enum types only had 3, 10 or 32 different values.

The type with the 32 different values actually only has 31 significant
bits since the enum constant OP_OPMASK is only used when querying the
enum, not for defining or describing the possible values. For this
reason, it was unavoidable to refactor the rtti macros, to support even
this case.
 1.470  24-Aug-2020  rillig make(1): in debug mode, print GNode details in symbols

A string like OP_DEPENDS|OP_OPTIONAL|OP_PRECIOUS is much easier to read
and understand than the bit pattern 00000089.

The implementation in enum.h looks really bloated and ugly, but using
this API is as simple and natural as possible. That's the trade-off.

In enum.h, I thought about choosing the numbers in the macros such that
it is always possible to combine two of them in order to reach an
arbitrary number, because of the "part1, part2" in the ENUM__SPEC macro.
The powers of 2 are not these numbers, as 7 cannot be expressed as the
sum of two of them. Neither are the fibonacci numbers since 12 cannot
be expressed as the sum of 2 fibonacci numbers. I tried to find a
general pattern to generate these minimal 2-sum numbers, but failed.
 1.469  23-Aug-2020  rillig make(1): remove redundant range restriction

re.re_nsub is at least 0, and (at least 0) + 1 cannot be 0.
 1.468  23-Aug-2020  rillig make(1): restructure code of the :ts modifier

Now there is one paragraph per case, as opposed to the highly condensed
code from before.

In the cases of :ts\n and :ts\t, the variables for number parsing are no
longer in the code path, which makes the code easier to understand for
humans. Not sure what the compilers made out of it.

The assignment "base = 0" has gone since it was redundant. An integer
that starts with 0 is always interpreted as octal by strtol, no matter
if the base is 8 or 0.
 1.467  23-Aug-2020  rillig make(1): extend the documentation on the ApplyModifier functions
 1.466  23-Aug-2020  rillig make(1): split code for parsing the :U modifier into digestible parts

The comment about "unescaped $'s that aren't before the delimiter" was
wrong, as the code didn't contain the "aren't before the delimiter"
part. By splitting the code into paragraphs, the larger structure
becomes easily visible. Having a few short comments in the right place
is more helpful than a big block of text.
 1.465  23-Aug-2020  rillig make(1): make brk_string return size_t for the number of words
 1.464  23-Aug-2020  rillig make(1): save a variable lookup when adding a read-only variable
 1.463  23-Aug-2020  rillig make(1): fix assertion failure in debug output for read-only variables
 1.462  23-Aug-2020  rillig make(1): improve variable name in ApplyModifiers

The bare q was too unspecific.
 1.461  23-Aug-2020  rillig make(1): use local variable in ApplyModifier_Path

Without this variable, it might seem as if st->newVal could become NULL
in some code paths.
 1.460  23-Aug-2020  rillig make(1): add assertions for non-nullness of strings

Verified by manually inspecting all call sites.
 1.459  22-Aug-2020  rillig make(1): clean up headers

Remove redundant headers that are already included by "make.h".
Make <assert.h> available to all compilation units that use "make.h".
 1.458  22-Aug-2020  rillig make(1): prevent assignment to the variable with the empty name
 1.457  22-Aug-2020  sjg Add .SHELL as read-only variable

The .SHELL variable represents the shellPath used to run
scripts.

Reviewed by: rillig, christos
 1.456  22-Aug-2020  rillig make(1): fix indentation
 1.455  22-Aug-2020  rillig make(1): fix documentation of Var_Parse
 1.454  20-Aug-2020  rillig make(1): make a few comments more precise
 1.453  20-Aug-2020  rillig make(1): use more descriptive variable name in ModifyWords
 1.452  20-Aug-2020  rillig make(1): remove VARP_SUB_MATCHED

This flag didn't really belong to the other flags. The other flags are
set during parsing and are then left as-is by ModifyWord_Subst and
ModifyWord_SubstRegex.

It's clearer to use a separate variable for storing whether there was a
match already.
 1.451  20-Aug-2020  rillig make(1): consistently access args->pflags in ModifyWord_Subst

It was confusing that some accesses were via pflags and some via
args->pflags.
 1.450  20-Aug-2020  rillig make(1): remove unreached code from bmake_strndup

The "at most" branch was never taken since all call sites in var.c only
ever need a substring, and the target buffer is not limited. Therefore
rename the function and make it simpler.

It's ok that bmake_strldup is defined as estrndup in case of USE_EMALLOC
since that function's implementation is compatible to the "copy
exactly", it just contains some extra null checks that will never match
since the variable values cannot (well, or should not) contain null
bytes. Theoretically they can, but the behavior then depends on the
exact implementation and is unreliable, therefore nobody does this.
After all, Makefiles are used for text processing, not for binary data.
 1.449  13-Aug-2020  rillig make(1): remove type alias Byte = char

This alias was only actually used in very few places, and changing it to
unsigned char or any other type would not be possible without generating
lots of compile-time errors. Therefore there was no abstraction, only
unnecessary complexity.
 1.448  12-Aug-2020  rillig make(1): prepare var.c for WARNS=6

I'm unhappy with the (unsigned) casts for the enum constants, but there
is no way around that since the base type of enums is int by definition,
and there is no way in C90 to explicitly specify the base type of an
enum type.

The (size_t) casts on the pointer differences are another pain point,
but for these as well, C90 does not have a ready-made solution. A
possible workaround would be an inline function ptr_diff that returns
size_t instead of ptrdiff_t, but that would make the code look even more
bloated than with the casts.

The remaining casts to (char), (int) and (time_t) look so suspicious
that it's easy enough to stumble upon them, as soon as someone finds the
time to fix these properly by adding a function ParseInt or
ParseSeconds.
 1.447  12-Aug-2020  rillig make(1): replace redundant NULL tests with assertions
 1.446  10-Aug-2020  rillig make(1): replace snprintf with concat3

This makes the code more predictable for long strings, since the main
action is never skipped.

The code also becomes simpler since there is no %s anymore, and the 3
arguments just appear in the same order as in the result.
 1.445  09-Aug-2020  rillig make(1): clean up code for subexpressions in the :C modifier

An ampersand in the replacement string can never produce an
out-of-bounds error or an undefined-subexpression error. This makes the
error message simpler since it only needs to cover the case of a single
digit.
 1.444  09-Aug-2020  rillig make(1): use char literals, improve documentation
 1.443  09-Aug-2020  rillig make(1): revert C99 initializer in ApplyModifiers

The code of usr.bin/make is supposed to be C90-compatible. That's why
it neither uses line-end comments nor declaration-after-statement nor a
few other features.

The fields that were copied "by name" did not get any additional
comments, only the ones with literal values did, since it's hard to see
what these mean.

This style of initializer has the additional benefit that a missing or
extraneous initializer-item would lead to a compile-time error, rather
than going undetected.
 1.442  09-Aug-2020  rillig make(1): mark inconsistent word separators for variable modifiers
 1.441  09-Aug-2020  rillig make(1): improve type signature of Var_Export
 1.440  09-Aug-2020  rillig make(1): extend the documentation for variable flags
 1.439  09-Aug-2020  rillig make(1): clean up indentation, includes, add documentation
 1.438  09-Aug-2020  rillig make(1): document a hard-to-understand pointer calculation
 1.437  09-Aug-2020  rillig make(1): restrict the type of startc and endc to char
 1.436  09-Aug-2020  rillig make(1): use fixed-size arrays for Enum_ToString

Declaring the ToStringSize as "static const size_t" made it an ordinary
integer expression. To avoid variable length arrays, the commonly
accepted way is to declare the length as an enum constant, as that is
considered an integer constant expression, which in turn makes the
declaration a fixed-size array.
 1.435  09-Aug-2020  christos Fix build; gcc does not like VLA's, so hard code the boundary
Use c99 initializers
 1.434  08-Aug-2020  rillig make(1): merge duplicate code for the :H :T :R :E modifiers

By setting modifyWord first, the rest of the code becomes exactly the
same and is merged by the compiler.
 1.433  08-Aug-2020  rillig make(1): remove trailing Z from buffer functions

This Z had been useful during the migration from int to size_t. This
migration is finished, at least for the Buffer type, so the Z is no
longer necessary.
 1.432  08-Aug-2020  rillig make(1): add debug logging for Var_Parse

This will hopefully help with tracking down why the "Undefined variable"
error message is not triggered.

There is some other non-obvious behavior nearby. In Parse_DoVar, the !=
assignment operator evaluates the variable using VARE_UNDEFERR, but
there is not even a warning for the following line:

!= echo err ${UNDEFINED} 1>&2
 1.431  08-Aug-2020  rillig make(1): extend debug logging to error values

The error marker var_Error is just an empty string. In the debug log
this empty string was not distinguishable from an otherwise empty
string.

Having this distinction helps in understanding the exact data flow.
 1.430  08-Aug-2020  rillig make(1): add more detailed debug logging for variable modifiers

Before a modifier is applied to a variable, it is not yet parsed,
therefore it is only possible to log a rough estimate of the modifier.
But after applying it, the parsing position has advanced, and the full
modifier can be logged.

In addition, to fully understand how the modifiers work, it's not enough
to just know the variable names and values, there are also some flags
that influence how the modifiers behave. The most influential is
VARE_WANTRES.

Thanks to sjg for the extensive review and valuable feedback on the
first drafts.
 1.429  08-Aug-2020  rillig make(1): improve documentation of Var_Parse and reorder conditions
 1.428  08-Aug-2020  rillig make(1): improve error message in case of unfinished modifiers

The previous error message "Unclosed substitution" was wrong for several
reasons.

It is not about "unclosed", but about "unfinished" since in the
:@var@...@ modifier the missing '@' does not really close anything.

The word "substitution" may have originated in a time where :S and
:from=to were the only modifiers, and these were indeed substitutions,
but several other modifiers aren't.

The :S and :C modifiers allow an arbitrary delimiter, therefore it is
helpful to enclose the delimiter in quotes, just in case someone chooses
')' or '{' or even ' ' as delimiter.
 1.427  08-Aug-2020  rillig make(1): align the code in Var_Append with Var_Set

The code has the same effect in both functions, therefore it should also
look the same.
 1.426  08-Aug-2020  rillig make(1): consistently use postfix-increment
 1.425  08-Aug-2020  rillig make(1): clean up VarStrftime

The parameter names were confusing and had outdated types.
 1.424  08-Aug-2020  rillig make(1): add test for unknown sub-modifier of the :S modifier
 1.423  08-Aug-2020  rillig make(1): clean up Var_UnExport

Mark the parameter as constant since it is not modified.

Remove tests for '\n' since these can never succeed.

newenv can never be NULL since neither of bmake_malloc or bmake_realloc
returns NULL.

Improve variable names: vlist was too unexpressive.

Add debug logging since unexporting variables is an uncommon operation
that directly affects the observable environment of the child processes.

Fix CRLF line endings in a few unit tests.
 1.422  08-Aug-2020  rillig make(1): clean up Var_Export

The string from the .export directive can never be NULL.

The test for local single-char variables is already done in Var_Export1.
 1.421  08-Aug-2020  rillig make(1): change return type of Var_Export1 to Boolean
 1.420  08-Aug-2020  rillig make(1): fix yoda-style comparisons in var.c
 1.419  08-Aug-2020  rillig make(1): condense comment for ApplyModifiers

This comment only repeats what is written in the manual page.
Furthermore, it didn't get updated consistently. The missing modifiers
were :_, :gmtime, :localtime, :hash, :Q, :range, :Or.
 1.418  08-Aug-2020  rillig make(1): fix debug output for indirect modifiers

The trailing part after the modifier was confusing at best. It does
make sense to log the string from p to p+rlen, but not from p+len to
p+len+len.

Consistently use double quotes for strings.
 1.417  07-Aug-2020  rillig make(1): format the code in var.c a little more consistently
 1.416  06-Aug-2020  rillig make(1): use consistent name for result of Cmd_Exec
 1.415  06-Aug-2020  rillig make(1): reduce code size in :D and :U modifier

This affects both the source code size and the binary code size, at
least on x86_64.
 1.414  06-Aug-2020  rillig make(1): clean up code for variable handling

The VarFlags type doesn't need an underscore. The underscore is
typically only used for exported functions.

Document the memory allocation details for Var.name.

Use consistent names for Hash_Entry variables. Quite possibly one of
them was a Lst_Node in ancient times, but that's not a reason to keep
this variable name forever.
 1.413  03-Aug-2020  rillig make(1): fix wrong assertion in ApplyModifiers

The assertion was supposed to validate startc and endc individually.

For unknown reasons (once again), this reduces the code size by 49 bytes
on x86_64.
 1.412  03-Aug-2020  rillig make(1): no declaration-after-statement anymore

NetBSD make is intended to be maximally portable, therefore it uses only
C89. This was not declared in the Makefile before.

There are still a few places in parse.c and metachar.c that use
end-of-line comments. These will be fixed in a follow-up commit.
 1.411  03-Aug-2020  rillig make(1): use VAR_DEBUG macro instead of if statements

This removes some clutter from the code and reduces the indentation.
 1.410  03-Aug-2020  rillig make(1): in lint mode, disallow dynamic variable names in :@ modifier

This is an extremely obscure feature that hopefully nobody ever
considered using.
 1.409  03-Aug-2020  rillig make(1): replace st->next with a simple string iterator

This is the same style that OpenBSD's make has been using for 20 years
now. Having one less field in ApplyModifiersState reduces complexity in
that place.

The individual modifiers currently don't handle parse errors
consistently. Some update pp while others leave it as-is. This
behavior may be aligned in a follow-up commit.
 1.408  03-Aug-2020  rillig make(1): don't use st->next as a local variable
 1.407  02-Aug-2020  rillig make(1): remove duplicate code in :ts modifier

The pointer xp had already been there with the correct value, and
there's no point in not using it. This reduces the code size on x86_64
by 48 bytes, which is more than I had expected.
 1.406  02-Aug-2020  rillig make(1): clean up NULL pointer comparisons, use separate variable

st->newVal is not meant to be a general-purpose storage.

Eliminate the unnecessary initialization of freeIt since Var_Parse
initializes it in every case.
 1.405  02-Aug-2020  rillig make(1): make :hash modifier more efficient

It's not necessary to allocate a 256-byte buffer just to store 9 bytes
in it, and to null-terminate it after writing every single byte to it.
 1.404  02-Aug-2020  rillig make(1): move qsort helper functions closer to ApplyModifier_Order
 1.403  02-Aug-2020  rillig make(1): merge duplicate code in the :O and :u modifiers
 1.402  02-Aug-2020  rillig make(1): inline VarOrder into ApplyModifier_Order

Removing the extra mapping layer "otype" makes the code both faster and
smaller.
 1.401  02-Aug-2020  rillig make(1): reduce scope of variables

This groups the variables by topic and only introduces them when they
are actually needed. The compiler doesn't care, but during a debugging
session this means fewer uninitialized variables.

In VarUniq, the variable i has been split into two. All others remain
exactly as before, just with a smaller scope.
 1.400  02-Aug-2020  rillig make(1): clean up comments in var.c

Some of the effects listed in "Side Effects" were really "Main Effects"
and have been moved to the main description.

VarAdd: There's no "front" in a hash table.

VarHash: A const char * is not a "String to modify".

Var_Subst: Even in r1.1 of var.c, there was no "variable structure to
store in" anywhere nearby.
 1.399  02-Aug-2020  rillig make(1): remove visual clutter from VarFind

Having fewer { else } around makes the return statements easier to find.
 1.398  02-Aug-2020  rillig make(1): remove unnecessary assignment in Var_Parse

In a variable expression without braces or parentheses, it is not
possible to have modifiers. Therefore the assigned endc could not have
been used anywhere.
 1.397  02-Aug-2020  rillig make(1): eliminate another unnecessary UNCONST in Var_Delete
 1.396  02-Aug-2020  rillig make(1): document almost duplicate code

ParseModifierPart and Var_Parse are very similar, but there might be
subtle differences. Until these differences are documented in the form
of unit tests, merging these code pieces is too dangerous.
 1.395  02-Aug-2020  rillig make(1): eliminate local variable in ParseModifierPart

The evaluation flags passed to the nested variables consist of 3 flags.
Therefore, (eflags & VARE_UNDEFERR) | (eflags & VARE_WANTRES) is
equivalent to (eflags & ~VARE_ASSIGN).

The variable name errnum was misleading anyway, just like the other
errnum from a few commits ago that had been renamed to errfmt.
 1.394  02-Aug-2020  rillig make(1): don't needlessly chain p-- and p++ in ParseModifierPart

At least GCC 5 didn't optimize this, although I wouldn't have been
surprised if it did.
 1.393  02-Aug-2020  rillig make(1): reduce the number of moving variables in ParseModifierPart

Having only the p walk through the string is easier to understand than
assigning between p and cp2 (with its unexpressive name).
 1.392  02-Aug-2020  rillig make(1): use shorter local variable names

The c in cp was redundant since the context makes it obvious that this
is a character pointer. In a tight loop where lots of characters are
compared, every letter counts.
 1.391  02-Aug-2020  rillig make(1): reduce indentation in ParseModifierPart
 1.390  02-Aug-2020  rillig make(1): restructure documentation of ParseModifierPart

Before, the long documentation looked too frightening, and it was not
immediately clear which parts of it had to be read and which could be
skipped.
 1.389  02-Aug-2020  rillig make(1): update implementation documentation
 1.388  01-Aug-2020  rillig make(1): switch Buffer size from int to size_t

This change helps to make the various integer types compatible and is a
preparational step for setting WARNS=6 in the Makefile.

The documentation of buf.c has been cleaned up and condensed since it
was mostly redundant, and some statements were even slightly wrong.

All code changes are covered by the existing unit tests, except for the
few lines in for.c around for_var_len. These changes have been reviewed
thoroughly and manually, like all the others in this commit.

Those buffer functions that deal with sizes have been renamed by
appending a Z, to make sure that no function call was accidentally
forgotten. They will be renamed back in a follow-up commit.

As usual, the scope of a few affected variables has been reduced, and
some variables had to be split since they had been incorrectly merged
before.

The order of the arguments to Buf_AddBytes has changed from (mem_len,
mem) to (mem, mem_len), in order to make it consistent with the
functions from the C standard library, such as snprintf.
 1.387  01-Aug-2020  rillig make(1): start nesting level in ApplyModifier_Match at 0

There is no need to start at 1, and starting at 0 generates smaller code
on x86_64. Reordering the --nest and the following break would increase
the code size though, for unknown reasons.
 1.386  01-Aug-2020  rillig make(1): inline and untangle the code for the :range modifier

There's no need to keep the result from brk_string in memory until the
buffer has been filled with the range. The only thing necessary from
brk_string is the number of words.
 1.385  01-Aug-2020  rillig make(1): fix error message when regcomp fails

If regcomp fails, it's not the regex substitution that is erroneous, but
the regex compilation.
 1.384  01-Aug-2020  rillig make(1): use enum for return values of Cond_Eval and friends
 1.383  01-Aug-2020  rillig make(1): fix documentation, use safer fallback value

The variable res is overwritten in every case, but better be safe than
sorry. In case of programming errors it's better to issue an error
message than to continue.
 1.382  01-Aug-2020  rillig make(1): make ModifyWords simpler

There is no need to test whether an actual word has been added. The
rule is simply "add a space before every word, except for the very
first".
 1.381  01-Aug-2020  rillig make(1): inline literal dollar characters

This reduces the code size when compiling with GCC 5. Apparently GCC
wasn't sure enough that str[0] and str[1] stay the same around the call
to Buf_AddByte.
 1.380  01-Aug-2020  rillig make(1): use consistent indentation in source code

Tabs for multiples of 8, then spaces.

The usage string has been kept as-is since the spaces there are
indentional and do influence the output.
 1.379  01-Aug-2020  rillig make(1): inline function in SysV modifier processing
 1.378  01-Aug-2020  rillig make(1): reduce the number of string comparisons in ${VAR:%.c=%.o}

There is only a single position in the word where the tail ".c" can
match, since it is implicitly anchored at the end. Therefore there's no
need to do several string comparisons.
 1.377  01-Aug-2020  rillig make(1): make variable names in SysV modifier more expressive

When matching a word against a substitution having a lhs and a rhs,
including a possible wildcard, there are just too many string variables
around to know what a simple "len" means and where it belongs.
 1.376  01-Aug-2020  rillig make(1): more descriptive variable names in Str_SYSVSubst
 1.375  01-Aug-2020  rillig make(1): let Var_Value return a const char *

The return value must not be modified anyway, so let the compiler check
this for free.
 1.374  01-Aug-2020  rillig make(1): fix documentation of Var_Value
 1.373  01-Aug-2020  rillig make(1): use better variable names in Var_Exists

Calling strchr just to initialize a variable to NULL is not as
straight-forward as possible.

The unspecific variable name cp made it unnecessarily difficult to
understand its purpose.
 1.372  01-Aug-2020  rillig make(1): make condition in Var_Export1 simpler to read
 1.371  01-Aug-2020  rillig make(1): use consistent pattern for testing flags
 1.370  01-Aug-2020  rillig make(1): clearly mark a variable in Var_Set_with_flags for freeing

No functional change.
 1.369  01-Aug-2020  rillig make(1): remove unnecessary snprintf from Var_ExportVars

String literals can be concatenated at compile time.

Furthermore, MAKE_EXPORTED can now contain percent characters without
invoking undefined behavior. Not that anyone would ever want to do this.
 1.368  31-Jul-2020  rillig make(1): mark unmodified part of ApplyModifiersState as constant
 1.367  31-Jul-2020  rillig make(1): reduce number of parameters to ApplyModifiers

This saves a call to strlen and some arithmetic operations.
 1.366  31-Jul-2020  rillig make(1): clean up Var_Parse

If *freePtr is not NULL, it only makes sense to free that pointer and not
a semmingly unrelated variable.
 1.365  31-Jul-2020  rillig make(1): remove some redundant parentheses
 1.364  31-Jul-2020  rillig make(1): omit unnecessary memory writes in :tu and :tl modifiers
 1.363  31-Jul-2020  rillig make(1): fix undefined behavior when parsing incomplete :t modifier

mod[2] must only ever be accessed if mod[1] != '\0'.
 1.362  31-Jul-2020  rillig make(1): reduce scope of local variables in ApplyModifier_Subst
 1.361  31-Jul-2020  rillig make(1): reduce scope of local variables in ApplyModifier_Regex
 1.360  31-Jul-2020  rillig make(1): fix undefined behavior when parsing malformed :C modifier

Same as in the :S modifier.
 1.359  31-Jul-2020  rillig make(1): fix undefined behavior in malformed :S modifier

The unit tests failed on Ubuntu, and by looking closely at the code, it
was trivial to see the out-of-bounds memory read.

Other modifiers may have the same problem and will be inspected later.
 1.358  29-Jul-2020  rillig make(1): remove unnecessary assignments to st->next

No change in the size of the resulting binary. Apparently GCC already
knew these assignments were redundant.
 1.357  29-Jul-2020  rillig make(1): document parameters to ApplyModifiers
 1.356  29-Jul-2020  rillig make(1): use specialized return type for ApplyModifier functions

This makes it immediately obvious what happens after a modifier has been
applied, instead of having to translate single-character mnemonics or
booleans to their actual intention.

This also reduces the size of the binary since there are fewer jumps.
 1.355  29-Jul-2020  rillig make(1): fix segfault when evaluating ${::=value}

The bug had been in the handling of the SysV modifier for many years, but
it had not been triggered since the "parsing position for the next
modifier" had been initialized to a non-NULL pointer.

In var.v r1.350, this pointer had been initialized to NULL instead since
every ApplyModifier function must set it in every case where it returns
anything except "default_case".

There might have been a slight chance of tricking make to output a wrong
error message, but nothing worse.
 1.354  29-Jul-2020  rillig make(1): add enough tests to cover the ApplyModifier functions

Only a few return statements are still missing from the code coverage.

In ApplyModifier_Assign, the test for an empty variable name is skipped
for now since it segfaults.

In ApplyModifier_SysV after the second ParseModifierPart, the branch for
the missing delimiter is not reached since this case is already checked
for in the first part of the function. To trigger this branch, a
specially crafted, unrealistic string needs to be created, and that's too
complicated for the moment.
 1.353  28-Jul-2020  rillig make(1): issue a parse error on unclosed variables

These errors had not been detected before by make.
In pkgsrc, this task as been done by pkglint since 2019-04-07.
 1.352  28-Jul-2020  rillig make(1): remove dead code from Var_Subst

The first parameter from Var_Subst had been a literal NULL in all cases.
These have been fixed using this command:

sed -i 's|Var_Subst(NULL, |Var_Subst(|' *.c

The one remaining case was not found because the "NULL," was followed by
a line break instead of a space.

The removed code probably wouldn't have worked as expected anyway.
Expanding a single variable to a literal string would have led to
unexpected behavior for cases like ${VAR:M${pattern}}, in case pattern
would contain an unescaped ':' itself.
 1.351  28-Jul-2020  rillig make(1): remove redundant ApplyModifiersState.termc
 1.350  27-Jul-2020  rillig make(1): document and reorder ApplyModifiersState
 1.349  27-Jul-2020  rillig make(1): rename ApplyModifiersState.nstr and newStr

The old names were too easy to confuse. In Var_Parse, nstr is indeed
used for the "new string", but that was misleading in ApplyModifiers,
especially since there was another variable called newStr.

The new variable names focus on the purpose of the variables, not on
their data types.
 1.348  27-Jul-2020  rillig make(1): rename ApplyModifiersState.cp to next

The name "next" is more descriptive than a mere "cp". It's easy to look
up the type of the variable, but not so easy to guess the purpose of the
variable, therefore the new name concentrates on the purpose.
 1.347  27-Jul-2020  rillig make(1): in ApplyModifier_Match, use short local variable name
 1.346  27-Jul-2020  rillig make(1): in ModifyWord_Match, use different variables

Using the same registers or memory locations for different variables is
the job of the register allocator, not the programmer. For humans, that
practice is rather confusing.
 1.345  27-Jul-2020  rillig make(1): replace comment in ApplyModifier_Match with code
 1.344  27-Jul-2020  rillig make(1): remove unused lengthPtr and freePtr from ApplyModifiersState
 1.343  27-Jul-2020  rillig make(1): remove unnecessary field from ApplyModifiersState
 1.342  27-Jul-2020  rillig make(1): remove unnecessary variable assignment

It is not the job of an ApplyModifier function to directly manipulate the
final result of the ApplyModifiers function. In fact, the resulting
pointer was overwritten unconditionally, in such an obvious way that the
compiler could have detected this. GCC 5 didn't though, which reduces
the code size a bit.
 1.341  27-Jul-2020  rillig make(1): move modifier character out of ApplyModifiersState

It is only used in the outer part of the loop to apply the modifiers.
 1.340  27-Jul-2020  rillig make(1): replace macros with functions

Having the hidden parameter st->endc in the macro made it unnecessarily
difficult to understand the code.
 1.339  27-Jul-2020  rillig make(1): use shorter variable name for parsing :D and :U
 1.338  27-Jul-2020  rillig make(1): remove unnecessary argument validation
 1.337  27-Jul-2020  rillig make(1): document API of Var_Value
 1.336  26-Jul-2020  rillig make(1): make test for dynamic variable name more specific

Since all the string literals have either 7 or 8 characters, there is no
need to call strcmp for shorter or longer variable names.
 1.335  26-Jul-2020  rillig make(1): extract code for dynamic variable names out of Var_Parse
 1.334  26-Jul-2020  rillig make(1): fix off-by-one error in Var_Parse for dynamic variables
 1.333  26-Jul-2020  rillig make(1): rename VarEvalFlags variables to eflags

This way, they cannot be confused with other flags, and any mismatch of
mixing eflags with constants other than VARE_* is immediately apparent.
 1.332  26-Jul-2020  rillig make(1): rename local variable in Var_Parse
 1.331  26-Jul-2020  rillig make(1): make return value of Var_Parse constant

This return value is not supposed to be modified since it can be a string
literal. The modifiable part is returned via freePtr, but only for
freeing, not for actually modifying anything.
 1.330  26-Jul-2020  rillig make(1): revert making Var_Parse return a const string

The dependencies between the C files and nonints.h are not declared
properly, which resulted in a successful partial build, but a full "make
clean && make" failed.
 1.329  26-Jul-2020  rillig make(1): mark result of Var_Parse as const

Var_Parse can return either a string literal or an allocated string. The
former must not be modified by the caller. If the string is allocated,
it is returned in freePtr as well, but only for freeing it after use.
 1.328  26-Jul-2020  rillig make(1): reduce scope and rename local variable in Var_Parse
 1.327  26-Jul-2020  rillig make(1): eliminate another local variable in Var_Parse
 1.326  26-Jul-2020  rillig make(1): split local variable into two in Var_Parse
 1.325  26-Jul-2020  rillig make(1): split local variable into two

Reusing a const char * parameter to store a char * and later free that
string was not a good idea. It made the pretty long code of Var_Parse
more difficult to understand.
 1.324  26-Jul-2020  rillig make(1): reorder arguments of ParseModifierPart

First the input parameters, then additional data, then special arguments.
 1.323  26-Jul-2020  rillig make(1): merge VARE_NOSUBST into VARE_WANTRES

The flag VARE_NOSUBST is only a few days old. It had grown out of the
VARP_NOSUBST and VAR_NOSUBST flags, not knowing at that time that it
meant the exact opposite of VARE_WANTRES.
 1.322  26-Jul-2020  rillig make(1): remove wrong comment from VarOrder
 1.321  26-Jul-2020  rillig make(1): reorder code in ParseModifierPart to be more readable
 1.320  26-Jul-2020  rillig make(1): in ParseModifierPart, only update length on success
 1.319  26-Jul-2020  rillig make(1): remove wrong comment for VarUniq

VarUniq is not about sorting the words.
 1.318  26-Jul-2020  rillig make(1): save a few bytes and cycles by comparing against 0

The comparison against ac - 1 could have been optimized by the compilers
as well, but both GCC 5 and Clang produce smaller code for the comparison
against 0.
 1.317  26-Jul-2020  rillig make(1): remove unnecessary block scope
 1.316  26-Jul-2020  rillig make(1): avoid memory allocation for oneBigWord in modifiers
 1.315  26-Jul-2020  rillig make(1): help the compiler to find common subexpressions
 1.314  26-Jul-2020  rillig make(1): add convenience functions for adding to a SepBuf
 1.313  26-Jul-2020  rillig make(1): add Buffer functions for common tasks

Adding a string or a substring to a buffer are common tasks when handling
variables. There is no need to spell out the strlen call or the pointer
subtraction at every call site.

Subtracting pointers results in a ptrdiff_t, which would have to be
converted to an int in each case for WARNS=6. Having this conversion in a
single place keeps the code clean.
 1.312  26-Jul-2020  rillig make(1): add Buf_AddInt to make the calling code simpler
 1.311  26-Jul-2020  rillig make(1): make code in the :S modifier simpler

This code path is covered by the "sequences of letters" in modmisc.mk.
 1.310  26-Jul-2020  rillig make(1): fix bug in :S modifier from 2020-07-19
 1.309  26-Jul-2020  rillig make(1): fix missing error message about missing @ delimiter

This had been broken since 2020-07-03, during a "refactoring".
 1.308  24-Jul-2020  rillig make(1): remove redundant condition

If there are multiple words from brk_string, none of them can be empty.
And if there is oneSingleWord that is empty, it doesn't hurt to add 0
bytes.
 1.307  24-Jul-2020  rillig make(1): prefer direct jump over indirect variable assignment
 1.306  24-Jul-2020  rillig make(1): make condition in ModifyWord_SubstRegex simpler

The generated code is still the same, at least with Clang.
 1.305  24-Jul-2020  rillig make(1): reduce lifetime of regmatch_t for :C modifier
 1.304  24-Jul-2020  rillig make(1): rename local variable
 1.303  24-Jul-2020  rillig make(1): allocate capturing groups for :C modifier statically

Since there are at most 10 capturing groups, it's easier to allocate them
on the stack.

To avoid growing the code size on x86_64, the matches have been moved to
the end of the struct, to keep the other offsets small.
 1.302  24-Jul-2020  rillig make(1): clean up Var_Export
 1.301  24-Jul-2020  rillig make(1): inline Var_Parse_State into ApplyModifiersState

There was no need to have this as a separate struct. Especially none of
the modifiers used by ModifyWords modify this state, therefore it was
not necessary to pass it by reference. This makes the code easier to
understand since there's less modifiable state.
 1.300  24-Jul-2020  rillig make(1): document ApplyModifiersState.cp
 1.299  24-Jul-2020  rillig make(1): eliminate ApplyModifiersState.tstr

This field always pointed to the beginning of the current modifier.
Therefore a better name is "mod", and there is no need to store it in the
shared state.
 1.298  23-Jul-2020  rillig make(1): only set missing_delim if it is actually missing

Just to make it easier to understand the code.
 1.297  23-Jul-2020  rillig make(1): fix wrong warning about missing delimiter
 1.296  23-Jul-2020  rillig make(1): reduce code size on x86_64
 1.295  21-Jul-2020  rillig make(1): remove parameter from ModifyWordCallback

Most modifiers don't need a variable context, and those that do already
get all their parameters in a custom Args struct.
 1.294  21-Jul-2020  rillig make(1): make implementation of the :C modifier simpler
 1.293  21-Jul-2020  rillig make(1): only ask for newline if :Q actually needs it

This simple change reduces the binary size of var.o by 96 bytes (Clang,
x86_64). Given that the code is still the same, just in a different
order, this is a bit surprising.
 1.292  21-Jul-2020  rillig make(1): clean up various things in var.c
 1.291  21-Jul-2020  rillig make(1): use consistent and descriptive names for callback functions
 1.290  21-Jul-2020  rillig make(1): remove unnecessary VarSelectWords_t struct
 1.289  20-Jul-2020  rillig make(1): extract :ts<separator> modifier into separate function
 1.288  20-Jul-2020  rillig make(1): clean up implementation of :[...] and other modifiers
 1.287  20-Jul-2020  rillig make(1): fix edge cases with $ at the end of the :@ modifier

In both parts of the :@ modifier, by passing &pflags to
ParseModifierPart, a final $ was interpreted as an anchor, which only
makes sense in the :S and :C modifiers.

This edge case is neither used by src nor by pkgsrc, except for the unit
tests that have been adjusted.
 1.286  20-Jul-2020  sjg Make DEBUG_HASH less of a fire-hose.

Reporting keys on every lookup is overkill unless
playing with a new HASH, so wrap in #ifdef DEBUG_HASH_LOOKUP
Also add some stats at the end so we can see
final size and max chain length - maxchain is a better
variable name than maxlen.
 1.285  20-Jul-2020  rillig make(1): make conditions work without pattern flags

The "pattern flag" VARP_NOSUBST was used to prevent evaluating
irrelevant conditions. Omitting the evaluation flag VARE_WANTRES has the
same effect.
 1.284  20-Jul-2020  rillig make(1): fix obscure edge case for ${VAR:e$=x}

The characters "$=" do not occur in either src or pkgsrc, therefore this
case is unlikely to occur in practice.
 1.283  20-Jul-2020  rillig make(1): move documentation for assignment modifiers
 1.282  20-Jul-2020  rillig make(1): prefer memcmp over strncmp
 1.281  20-Jul-2020  rillig make(1): fix type of VarPattern.lhsLen and rhsLen

String length is measured in size_t, not int.
 1.280  20-Jul-2020  rillig make(1): fix undefined behavior in :S modifier

The expression word + wordLen - leftLen had resulted in an out-of-bounds
pointer before. Luckily the heap addresses were high enough in typical
applications to prevent a wrap-around.
 1.279  20-Jul-2020  rillig make(1): make implementation of the :S modifier simpler
 1.278  20-Jul-2020  rillig make(1): make modifier handling simpler

Implementing a modifier such as :S or :M should not be concerned with
separating the words of the resulting string. Ideally this should be
done in the same way by all modifiers.

Before, the :R (filename root) modifier added a separator even if the
resulting filename root was an empty string. The chances that this
change in behavior breaks anything are epsilon.

The :@ modifier, if it appeared after a :ts modifier, did not use the
word separator from the :ts modifier (which all other modifiers do) but
always added a space. This behavior has been preserved for now. It's an
unnecessary inconsistency though.

In contrast to Buffer, the newly added SepBuf uses size_t for memory
sizes and also uses the conventional parameter order (mem, memsize)
instead of the unusual (memsize, mem).
 1.277  19-Jul-2020  rillig make(1): remove unnecessary UNCONST
 1.276  19-Jul-2020  rillig make(1): make ampersand in ${VAR:from=to&} an ordinary character

In SysV substitutions, wildcards are expressed with % instead of &. The
& is not mentioned in the manual page, and having another wildcard for
the whole word would be such an obscure feature that not even pkgsrc uses
it. The easiest way to discover this feature had been to read the source
code of make(1) or to use a fuzzer and accidentally stumble upon this
edge case.
 1.275  19-Jul-2020  rillig make(1): don't call VarSubstitute unnecessarily
 1.274  19-Jul-2020  rillig make(1): remove wrong VarPattern from ApplyModifier_Exclam
 1.273  19-Jul-2020  rillig make(1): remove wrong VarPattern from ApplyModifier_Assign
 1.272  19-Jul-2020  rillig make(1): flatten ApplyModifier_Assign
 1.271  19-Jul-2020  rillig make(1): fix off-by-one error in :C modifier

Previously this off-by-one error had invoked undefined behavior.
Until today there was no corresponding unit test though.
 1.270  19-Jul-2020  rillig make(1): fix double free introduced this morning in r1.258
 1.269  19-Jul-2020  rillig make(1): remove redundant loop control variable
 1.268  19-Jul-2020  rillig make(1): rename ApplyModifiersState.flags to eflags

There are just too many variables called "flags" in this file, and their
constants must not accidentally overlap.
 1.267  19-Jul-2020  rillig make(1): clean up VarLoop structure
 1.266  19-Jul-2020  rillig make(1): remove unused parameter from ParseModifierPart
 1.265  19-Jul-2020  rillig make(1): rename VarGetPattern to ParseModifierPart
 1.264  19-Jul-2020  rillig make(1): eliminate unused local variables
 1.263  19-Jul-2020  rillig make(1): eliminate macro from VarGetPattern
 1.262  19-Jul-2020  rillig make(1): replace ApplyModifiersState field with local variable
 1.261  19-Jul-2020  rillig make(1): rename flags for :S and :C pattern matching

Renaming these flags makes it obvious that VarPattern is used in a few
places where it doesn't belong. These will be fixed in follow-up
commits.
 1.260  19-Jul-2020  rillig make(1): convert VarFind flags to typed enum
 1.259  19-Jul-2020  rillig make(1): rename Varf_Flags to VarEvalFlags

In var.c there are lots of different flag types. To make any accidental
mixture obvious, each flag group gets its own prefix.

The only flag group that is visible outside of var.c is concerned with
evaluating variables, therefore the "e", which replaces the former "f"
that probably just meant "flag".
 1.258  19-Jul-2020  rillig make(1): clean up handling of :? modifier

The VarPattern type had nothing to do with parsing conditions.
 1.257  19-Jul-2020  rillig make(1): clean up code in var.c

* use canonical comparison to zero in more cases
* omit unnecessary snprintf for string literal concatenation
* omit a few redundant parentheses and braces
 1.256  19-Jul-2020  rillig make(1): move SysV string matching to var.c

This kind of string matching is only used in variable modifiers, and only
if this feature is enabled by SYSVVARSUB.
 1.255  04-Jul-2020  rillig make(1): expand the test for variable expansion in the :@ modifier
 1.254  04-Jul-2020  rillig make(1): add debugging output for :@ modifier

The inner working of the :@ modifier is quite tricky. To understand what
really happens, it helps to look at each word as it is being modified.
 1.253  04-Jul-2020  rillig make(1): constify VarModify parameter

Since var.c r1.238 and r1.239, the callbacks for the :H :T :E :R modifiers
don't modify the word anymore, and the others didn't modify it at all.
 1.252  04-Jul-2020  rillig make(1): fix :hash modifier on 16-bit platforms

On platforms where int has only 16 bits the shifts would have been in
16-bit arithmetic, which would invoke undefined behavior for "ustr[3] <<
24" as well as "ustr[2] << 16" (C99, 6.5.7p3).
 1.251  04-Jul-2020  rillig make(1): remove unnecessary _t suffix from VarLoop type
 1.250  04-Jul-2020  rillig make(1): fix documentation of Var_Subst, clean up other documentation
 1.249  04-Jul-2020  rillig make(1): remove unnecessary forward declarations, fix indentation
 1.248  04-Jul-2020  rillig make(1): remove unnecessary forward declarations
 1.247  04-Jul-2020  lukem make: fix sign-compare warning

Fix -Wsign-compare warning on amd64, introduced in rev 1.239
 1.246  03-Jul-2020  rillig make(1): reduce line length in var.c
 1.245  03-Jul-2020  rillig make(1): refactor ApplyModifier_SysV to have less indentation
 1.244  03-Jul-2020  rillig make(1): clean up documentation, small refactorings for variables

- document the callback for VarModify
- clearly mark the callbacks
- shorten the documentation for some callback functions
- fix the documentation of VarSYSVMatch
- remove unnecessary null check from VarLoopExpand
- add test for applying modifiers to empty strings
 1.243  03-Jul-2020  rillig make(1): fix bug in :E modifier, introduced in r1.239 today

That case was not covered by the tests before.
 1.242  03-Jul-2020  rillig make(1): fix indentation in var.c

Fixing the indentation manually is a tedious task. indent(1) was no help
since it has several bugs and is not flexible enough for the original
make code.
 1.241  03-Jul-2020  rillig make(1): add Hash_ForEach to avoid duplicate code
 1.240  03-Jul-2020  rillig make(1): fix indentation and alignment
 1.239  03-Jul-2020  rillig make(1): make code for the :E and :R modifiers simpler
 1.238  03-Jul-2020  rillig make(1): clean up code for the :H and :T modifiers
 1.237  03-Jul-2020  rillig make(1): add missing test for the :H and :T modifiers
 1.236  03-Jul-2020  rillig make(1): refactor ApplyModifiers

Previously, ApplyModifiers was 1200 lines long and pretty complicated.
Now each of the complicated modifiers has its own function, which reduces
the indentation and makes it possible to use early returns.
 1.235  03-Jul-2020  rillig make(1): remove trailing whitespace
 1.234  03-Jul-2020  rillig make(1): remove redundant parentheses around return values
 1.233  02-Jul-2020  rillig make(1): fix unnecessary evaluation of variables in :@var@ modifier
 1.232  02-Jul-2020  rillig make(1): fix unnecessary evaluation for :S :C := modifiers
 1.231  02-Jul-2020  rillig make(1): clean up code in var.c

- remove redundant parentheses in return statements
- add missing space after comma
- remove some redundant parentheses around comparisons
- use hexadecimal numbers for bit mask constants
- remove Var_GetTail and Var_GetHead, which had been unused for 19 years
 1.230  02-Jul-2020  rillig make(1): remove useless parameter from Var_Set

The enum corresponding to this int parameter is only defined in var.c,
which makes it impractical for the outside to set this parameter to
anything but 0.

On x86_64, this reduces the size of the resulting executable by 5 kB.
 1.229  02-Jul-2020  rillig make(1): use enum instead of #define for VarPattern flags
 1.228  02-Jul-2020  rillig make(1): add more type safety for enums

There are several types of flags for variables, and these cannot be
mixed. To prevent accidental typos, these are defined in separate enum
types. Clang warns about direct assignments between distinct types, but
not about mixing distinct types in binary expressions like A | B. GCC
does not warn at all.
 1.227  02-Jul-2020  rillig make(1): fix unnecessary evaluation of :M and :N modifiers
 1.226  02-Jul-2020  rillig make(1): fix edge case for evaluating unnecessary conditions
 1.225  01-Jul-2020  sjg Fix parsing of nested variables during .for loop

Recent change to cond.c to avoid eval of unnecessary
terms had side effect on constructs like:

.for s in 1 2
.if defined(MAN$s) && !empty(MAN$s)
MAN+= ${MAN$s}
.endif
.endfor

resulting in MAN being flagged as recursive.

When Var_Parse encounters a variable within a variable name
we want to force it to be expanded.
But given the way get_mpt_arg calls Var_Parse we need to check
whether we actually started parsing a variable yet.
 1.224  05-Jun-2020  sjg make: add :Or for reverse sort

:Or is more efficient than :O:[-1..1]

Reviewed by: christos
 1.223  25-Apr-2020  christos - Percent on the rhs is special only if the lhs has one too.
- If the rhs does not have percent, but the lhs has, the result is the rhs
This behavior matches gmake
 1.222  03-Feb-2019  mrg - add or adjust /* FALLTHROUGH */ where appropriate
- add __unreachable() after functions that can return but won't in
this case, and thus can't be marked __dead easily
 1.221  21-Dec-2018  sjg unexport-env: avoid SEGFAULT if MAKELEVEL not in env
 1.220  27-May-2018  christos branches: 1.220.2;
- Introduce :q modifier for make variables and make it double escape $'s so
that passing variables to recursive makes with :q works as expected.
- Revert :Q to work as before.
- Adjust makefiles that use recursive make to use :q

Discussed on tech-toolchain@
XXX: pullup 8
 1.219  24-May-2018  christos Since ${MAKE} converts $$ -> $ during parsing we need to put it back to
preserve the original variable value with :Q.
XXX: pullup-8
 1.218  18-Feb-2018  sjg branches: 1.218.2;
Var_Set: avoid SIGSEGV if val is NULL

A NULL val is handled gracefully (by VarAdd) when
var is not previously set, so we ought not crash
the second time.

PR: 53034
 1.217  08-Dec-2017  sjg Ensure VAR+=val on command line is handled correctly

If VAR is not previously set, call Var_Set to deal with
the special case of VAR_CMD.

If VAR is previously set, and ctxt is VAR_CMD we should do the update
even if VAR_FROM_CMD is set.
 1.216  18-Nov-2017  sjg Do not append to variable set on command line

POSIX requires that variables set on the command line
be immutable.
Var_Append needs to pass FIND_CMD and skip append
if found variable has VAR_FROM_CMD flag set.
 1.215  16-Apr-2017  riastradh branches: 1.215.4;
Plug memory leak.

CID 978372
 1.214  16-Apr-2017  riastradh Use, don't kludge, MAKE_ATTR_UNUSED.

CID 1300234
CID 1300237
CID 1300238
CID 1300245
CID 1300255
CID 1300267
CID 1300284
 1.213  01-Feb-2017  sjg Since we are avoiding VAR_INTERNAL, allow the variable :_ stores to
to be specified, also allows for multiple stages of modification to
be stashed.
 1.212  01-Feb-2017  sjg Just have :_ use current ctxt, avoid need to force VAR_INTERNAL.
 1.211  31-Jan-2017  sjg Use VAR_INTERNAL when calling ApplyModifiers for VAR_{CMD,GLOBAL} only.
 1.210  30-Jan-2017  sjg Add :range and :_

:range replaces var value with an integer sequence one per word
in the current var value.

:_ stores the current var value in $_ so that it can be referred to
later in the modifier series.

Reviewed by: christos
 1.209  14-Jan-2017  sjg branches: 1.209.2;
Allow providing a utc value to :{gm,local}time

Reviewed by: christos
 1.208  03-Jun-2016  sjg branches: 1.208.2;
Add cached_realpath()

realpath(3) is expensive, and meta mode at least uses it extensively.
We use cached_realpath() to save the result of successful calls to
realpath(3) in a private variable context.

This improves the worst case performance (eg examining libc with
nothing to do) by a factor of 4.

Reviewed by: christos
 1.207  11-Mar-2016  matthias $% is $(.MEMBER) and not $(.ARCHIVE), $! is $(.ARCHIVE) and not $(.MEMBER)
 1.206  07-Mar-2016  sjg For :ts numeric escapes \x* is hex, anything else is octal.
 1.205  20-Feb-2016  sjg Add a knob .MAKE.SAVE_DOLLARS to control the behavior of $$ during :=

If TRUE '$$' are not consumed (saved).
If FALSE '$$' becomes '$' just like normal expansion rules.

default is TRUE.

Reviewed by: christos
 1.204  18-Feb-2016  sjg Delete :@ loop iterator when done with it
 1.203  18-Feb-2016  sjg Add .export-literal to avoid the need for $$ dance when trying to put
unexpanded variables into environment.

Reviewed by: christos
 1.202  18-Feb-2016  christos Collapse the 3 boolean parameter to 1 flags parameter. No functional change.
 1.201  09-Jan-2016  christos Preserve $$ in := assignments..

FOO=\$$CRAP
BAR:=${FOO}

all:
echo ${FOO}
echo ${BAR}
 1.200  01-Dec-2015  sjg Avoid calling brk_string() and hence Var_Export1() on
empty strings.
 1.199  20-Oct-2015  sjg Ensure emsg initialized
 1.198  12-Oct-2015  sjg The conditional expressions used with ':?' can be expensive
eg. exists() does stat(2).
If 'wantit' is FALSE, we are going to discard everything anyway,
so skip evaluating the conditional and expanding either lhs or rhs.
 1.197  11-Oct-2015  sjg Add Boolean wantit to Var_Parse and Var_Subst

wantit will be FALSE when we are just consuming to discard
in which case we skip "expensive" things like Cmd_Exec.

Reviewed by: christos
 1.196  06-Oct-2015  christos don't check for NULL before free() (Tilman Sauerbeck)
 1.195  19-Jun-2015  christos use isspace()
 1.194  19-Jun-2015  mlelstv Adjust metachar handling to previous behaviour:
- space and tab are no shell metachars, remove them from generic
metachar function
- add space and tab as to-be-quoted characters for :Q modifier
- add = and : as characters that require command handling by the shell
 1.193  17-Jun-2015  christos Centralize the "is a meta char" test, instead of using two different arrays.
 1.192  05-May-2015  sjg When evaluating condtionals from .if we want to require
that the lhs is a variable reference, a number or a quoted string.
This helps avoid subtle bugs caused by typos.

When conditionals are being evaluated during variable expansion
we cannot be as strict becuase lhs will already have been expanded.

We therefor pass a boolean to Cond_EvalExpression to tell it how
lhs should be treated.

Add unit-tests/cond2.mk to test the above

Reviewed by: christos, joerg
 1.191  14-Sep-2014  dholland Fix handling of the silly $(@D) $(@F) etc. vars so modifiers on them
work. This is issue #1 in PR 49085 from Jarmo Jaakkola, but I've used
a different and cleaner patch this time.
 1.190  13-Sep-2014  dholland Recognize the crazy POSIXisms $(?D) and $(?F); issue 2 in PR 49085
from Jarmo Jaakkola.
 1.189  13-Sep-2014  dholland typo in comment, from the first reverted patch
 1.188  07-Sep-2014  joerg Revert all make changes except the unit tests to the state of three
weeks ago. Individual changes can be reapplied after review.
 1.187  23-Aug-2014  christos PR/49085: Jarmo Jaakkola: fix several parsing errors

Don't exit from var.c:Var_Parse() before possible modifiers are handled
on D and F modified versions of local variables. Properly expand $(?D)
and $(?F) too.

Make line continuations in rule's commands POSIX compliant.

Fix the syntax error caused by lib(member) as the last target before
a dependency operator.

Document the line continuation change in the manual page. Also talk
more about the POSIX style local variables and their modifiers.

Add tests covering the fixed problems into d_posix.mk. The test is
a known failure at the moment because of PR 49086 and PR 49092.

[XXX: unconverted tests]
 1.186  20-Jun-2014  sjg Return varNoError rather than var_Error for ::?= etc.
 1.185  19-May-2014  sjg Var_Parse: endc only counts when at the correct nesting depth.
This ensures we correctly detect errors like:
.if empty(VAR && !empty(FOO)

Var_Subst: if Var_Parse returns var_Error it is an error.
This ensures we detect errors like:

VAR:= ${FOO.${GOO}
 1.184  04-Sep-2013  sjg branches: 1.184.2;
Add VAR_INTERNAL as a context for variables set by make itself,
which should not override those set by makefiles.
Currently MAKEFILE is the only variable affected.

Reviewed by: christos
 1.183  16-Jul-2013  sjg When a var is set in the CMD context, it prevents the same name
being set in GLOBAL context. We should also delete any such
variable in GLOBAL context, else make -V will show the wrong value.
 1.182  16-Jul-2013  christos More gmake compatibility:
1. add -w flag to print Entering and Leaving directory name the the beginning
and the end of processing.
2. export MAKELEVEL=$((MAKELEVEL + 1)) only in the child environment.
3. when printing error messages, prefix them with the program name [$MAKELEVEL]
for $MAKELEVEL > 0
4. if $MAKEFLAGS consists only of letters assume it is a set of flags (as
allowed by posix), convert them to -f -l -a -g -s, so that they get parsed
properly.
With those fixes gmake -> bmake -> gmake -> bmake etc. works as expected.
 1.181  15-Jul-2013  christos Make this work again like gmake. sjg: see the test program.
 1.180  06-Jul-2013  sjg If using gmake's MAKELEVEL; use it the same way
 1.179  18-Jun-2013  sjg Use a #define for the variable name we put in environment to pass .MAKE.LEVEL
in case we don't want to use gmake's MAKELEVEL in a different way.
 1.178  10-Jun-2013  christos put back previous change
 1.177  10-Jun-2013  joerg Revert last, it doesn't even build.
 1.176  10-Jun-2013  christos 1. Don't export .MAKE.LEVEL, because this is an illegal variable name for
some shells; export MAKELEVEL like gmake(1) does.
2. It is absurd for the environment variable to be one greater than the
make variable!?!?! To wit...
printf 'all:\n\t@echo ${.MAKE.LEVEL}; printenv .MAKE.LEVEL' | make -f -
 1.175  29-May-2013  sjg Clean up some left overs
 1.174  18-May-2013  sjg Var_Delete: expand name if needed.
 1.173  24-Feb-2013  christos While it is nice to have the same function do both the upper case and
lower case conversion, it ends up eating 4% of the total time we spend
in make calling tolower() while building libc.
 1.172  15-Nov-2012  christos include the variable name in debugging printfs.
 1.171  12-Jun-2012  joerg branches: 1.171.2;
Replace __dead, __unused and the various printf format attributes
with versions prefixed by MAKE_ATTR_* to avoid modifying the
implementation namespace. Make sure they are available in all places
using nonints.h to fix bootstrap on Linux.
 1.170  04-Jun-2012  sjg Var_UnExport() and setenv() can both realloc environ.
Use a common variable (savedEnv) to track that to avoid wasting memory.
Also, if providing setenv and unsetenv, do getenv too to ensure a consistent
set.
 1.169  18-May-2012  sjg Avoid nested extern declaration warnings
 1.168  24-Apr-2012  sjg Var* are generally very liberal with memory, with the expectation
that none of it persists for long.
This isn't always true - for example a long running .for loop.

Buf_DestroyCompact() is used by Var_Subst(), rather than Buf_Destroy().
If it looks like we can save BUF_COMPACT_LIMIT (128) or more bytes,
call realloc. This can reduce memory consumption by about 20%
Setting BUF_COMPACT_LIMIT to 0 dissables this.
 1.167  03-Jun-2011  sjg branches: 1.167.2;
ApplyModifiers: if we parse a variable in the modifier list,
do not consider it a nested modifier set, unless it is followed by
endc, a ':' or end of string.
 1.166  21-May-2011  tsutsui branches: 1.166.2;
Explicitly #include <time.h> for time(3) and strftime(3).
Fixes warnings on some non NetBSD hosts.
 1.165  11-Apr-2011  sjg If a "long" modifier name does not match, goto default case
so SysV modifier can be tried.
 1.164  11-Apr-2011  sjg Add :localtime and :gmtime which use value as format string for strftime.
 1.163  07-Apr-2011  joerg Add the :hash modifier to compute a 32bit hash of an variable.
This uses MurmurHash3 to get a reasonable collission-free hash with
small code. The result is endian neutral.
 1.162  06-Mar-2011  sjg Add unit tests to check that exists(some/dir/) works

Also fix handling of sysV substitutions when lhs and variable are empty.

Also that modifiers do not cause errors during conditional tests
when undefined variables should otherwise be ok.
Ie. .if defined(nosuch) && ${nosuch:Mx} != ""
 1.161  02-Dec-2010  christos From Aleksey Cheusov: Fix unexpected behavior with:
all: test1 test2
test1:
@echo ${foo::=foo-text}
test2:
@echo $(foo::=foo-text)
Saying:
nbmake: Unclosed substitution for foo (} missing)
for test 2
 1.160  02-Dec-2010  christos add commented out a possible addition that provides $^ like gmake, but
I am not sure if we should add it, since we already have $> for it.
 1.159  06-Jun-2010  sjg Add .export-env which tells make to export a variable to the environment
but not to track it - as is done for .export
This allows the variable to be updated without affecting what was put
into the environment.
Older versions of make will simply treat this as .export
 1.158  21-Apr-2010  sjg If we do .export (all) and have any variables that involve :sh
we will hit an error (var is recursive) while trying to evaluate that.
Fix, and add a unit test for this.
 1.157  20-Apr-2010  sjg Behavior of realpath() appears to vary.
To ensure consistent results, use stat(2) as a final check for success.
 1.156  15-Apr-2010  sjg Add :tA to attempt to resolve to absoute path using realpath().
 1.155  19-Nov-2009  sjg Add .unexport - the exact opposite of .export
and .unexport-env which unexport's all previously .export'd globals
as well as clearing environ[].
Allow's sys.mk near total controll.

Reviewed by: apb
 1.154  08-Sep-2009  sjg Reviewed by: apb

Use .MAKE.LEVEL to track recursion.
The first instance of make will have .MAKE.LEVEL 0, which
can be handy for excluding rules which should not apply
in a sub-make.
gmake and freebsd's make have a similar mechanism, but each
uses a different variable to track it. Since we cannot be
compatible with both, we allow the makefiles to cope if they want
by handling the export of .MAKE.LEVEL+1 in Var_Set().
 1.153  07-Sep-2009  sjg PR: 41998

:Ufu\:goo

should expand to fu:goo even when it appears in :M${:Ufu\:goo}
When scanning for :M do not compress \: if we know we have
to call Var_Subst.
 1.152  16-Jun-2009  sjg Child of vfork() must not alter the state of parent.
There is no need to touch the state of vars in child anyway.
Change 2nd arg to Var_Export1() to indicate if we are the parent or child,
and only set flags in the parent.
 1.151  18-Feb-2009  christos adjust patch to current.
 1.150  18-Feb-2009  christos CID 5044: Protect against NULL deref.
 1.149  28-Jan-2009  dsl branches: 1.149.2;
Allow for () in the argument to .ifdef et al.
Save/restore current values of global data across expression evaluation
to give ${foo:? ...} a change of working inside another .if.
 1.148  23-Jan-2009  dsl Change 'ClientData' to 'void *' so that relevant parameters can
be made 'const void *'.
 1.147  18-Jan-2009  lukem fix -Wsign-compare issues
 1.146  17-Jan-2009  dsl Change 'Buffer' so that it is the actual struct, not a pointer to it.
Saves having to malloc/free a fixed size structure.
Buf_Init() now takes ptr to Buffer to initialiase.
Change Buf_Destroy() to return ptr to string when not freed.
Remove large number of casts to (Byte) and (Byte *) - 'Byte' is 'char' here.
Buf_AddByte[s] guarantees that the data is 0 termininated, so never add '\0'.
Keep 'count' not 'left' and 'inPtr', code is simplier with only one update.
Fix fallou, no functional change.
 1.145  16-Jan-2009  dsl Debug print improvements.
 1.144  29-Dec-2008  dsl In the :M code, 'pattern' is always malloced memory - so unconditionally
free.
 1.143  29-Dec-2008  dsl Count both () and {} when looking for the end of a :M pattern.
In particular inside .if empty(...) it was only counting ().
In reality this needs further changes.
This may well fix recent pkgsrc breakage.
 1.142  23-Dec-2008  dsl Treat '\\' as a possible terminator for a loop control variable.
(Typically an escaped \: of a nested modifier.)
 1.141  21-Dec-2008  dsl Don't allow $) $} $: or $$ as variable expansions, they confuse other
parts of make that try to scan through variable expansions.
(given the make syntax that is all rather doomed anyway!)
Move the check for $\0 to a place where it will be detected - otherwise
the parser will run off the input buffer!
 1.140  20-Dec-2008  dsl A lot of code assumes that the pointer returned by Buf_GetAll() is malloced.
Replace Buf_Discard() with Buf_Empty() since the former might leave the
'outPtr != buffer' and the only caller wanted all the data discared.
Remove 'outPtr' now that it always equals 'buffer'.
The assumption about Buf_GetAll()is now guaranteed by design.
 1.139  20-Dec-2008  dsl Do not allow Var_Set() or Var_Append() to set the empty variable (which will
now forever remain undefined).
Rework/simplify code to avoid UNCONST().
 1.138  20-Dec-2008  dsl Change some "^I *^I" to "^I^I" and align the comment block about
variable modifiers.
 1.137  19-Dec-2008  christos add a little more debugging.
 1.136  13-Dec-2008  dsl Use NULL instead of -1 cast to the relavant type (usually via NIL).
This was a suggestion from christos - so blame him if there is a deep
reason for using -1 :-)
 1.135  13-Dec-2008  dsl Do not let the variable name "" be set from the command line (-D "")
or from ::= modifiers (${::=foo}).
 1.134  06-Oct-2008  joerg Don't use emalloc and friends directly, but call them consistently
bmake_malloc and friends. Implement them via macros for the native case
and provide fallback implementations otherwise. Avoid polluting the
namespace by not defining enomem globally. Don't bother to provide
strdup and strndup, they were only used for the estrdup and estrndup
comapt code.

This addresses the presence of emalloc in system libraries on A/UX and
resulted strange issues as reported by Timothy E. Larson.
 1.133  31-Jul-2008  joerg Micro-optimise VarQuote by exploiting that meta characters are normally
rare and skipping over normal characters with strcspn is therefore
faster. Further simplify logic by always setting newline and precompute
the size of the string.

OK christos@
 1.132  31-Jul-2008  joerg Meta characters are a constant, mark it so.
 1.131  15-May-2008  sjg VAR_CMD context is used by conditionals and other purposes, so
actually set VAR_FROM_CMD when appropriate and only skip setting in
VAR_GLOBAL when that flag is set.
 1.130  15-May-2008  sjg Disable this for now, VAR_CMD is abused for other purposes...
 1.129  14-May-2008  sjg Do not set vars in VAR_GLOBAL context, if they have been set on the
command line.
Add a suitable unit-test.
 1.128  31-Mar-2008  sjg branches: 1.128.2; 1.128.4;
If the name passed to Var_Exists contains '$' run it through Var_Subst.
 1.127  15-Feb-2008  christos back all changes out until I fix it properly.
 1.126  15-Feb-2008  christos remove error/noerror const
 1.125  15-Feb-2008  christos - symbolic names for error and no error
- more size_t
 1.124  14-Feb-2008  christos - use pid_t/size_t as appropriate instead of int.
- use %ld to print pids.
- fix a bit of lint.
- WARNS=4
 1.123  13-Oct-2007  apb Don't write into const strings; not even if we intend to
reinstate the original contents later.
 1.122  13-Oct-2007  apb * Convert all uses of strdup() to estrdup();
* Use estrndup() in a few cases where it simplifies the code;
* Provide compatibility definitions of strndup and estrndup;
 1.121  13-Oct-2007  apb Oops, missed a few more comments in previous.
 1.120  13-Oct-2007  apb Correct documentation for Var_Parse's freePtr arg. It's a pointer to
the thing to be freed (or NULL if nothing to free), not to a pointer to
boolean saying whether or not to free the result.
 1.119  09-Oct-2007  sjg Fix an off-by-one error in handing mal-formed modifiers.
The issue seems to have been present for some time, only showed up
when running unit-tests on SunOS.
Make sure we get an error message, but otherwise behave as before.
 1.118  05-Oct-2007  sjg Add the ability to .export variables to the environment.
 1.117  16-Jun-2007  dsl branches: 1.117.4;
Under DEBUG(VAR) report when the variable wasn't found.
Disposes of pr bin/15842
 1.116  01-Jan-2007  dsl Since the 'buf' code here is only used by make, we don't need two names
for the default buffer size of 256.
Also allow for the trailing 0 byte!
 1.115  27-Oct-2006  dsl branches: 1.115.2; 1.115.4;
Since 'ClientData' is 'void *', nuke almost all the (ClientData) casts.
 1.114  15-Oct-2006  dsl Output all debug trace output through 'debug_file' defaulting to 'stdout'.
(Almost all the debug output went there, but some went to stderr.)
Split the parsing of -d (debug flags) out into its own routine.
Allow the output filename to be changed by specifying -dF<file> to create
a log file, or -dF+<file> to append to it. <file> may be stdout or stderr.
Also change so that -d-<flags> acts on <flags> locally but doesn't copy
them to MAKEFLAGS so they aren't inherited by child makes.
I'm not 100% happy with the command line syntax for the above, so they are
currently undocumented.
 1.113  26-Aug-2006  christos More programs using efun.
 1.112  28-Jul-2006  sjg Indicate which modifier a result applies to, helps debug complex cases.
 1.111  29-Jun-2006  rillig Fixed the bug reported in PR 33866, which is that the :Q operator does not
handle newlines correctly. Ok'ed by christos.
 1.110  19-May-2006  christos branches: 1.110.2;
Coverity CI D3758: Plug memory leak.
 1.109  19-May-2006  christos Coverity CID 3757: Plug memory leak.
 1.108  11-May-2006  sjg Extract the variable modifier logic to a separate function.
This cuts Var_Parse in half! and allows the modifier logic to
be used recursively - when getting modifiers via variables.

Add new unit-test, to check that certain error cases are handled
correctly.
 1.107  22-Apr-2006  christos Coverity CID 529: Call VarFreeEnv to prevent leak.
 1.106  22-Apr-2006  christos Coverity CID 534: Free junk variables on return from Var_Parse.
 1.105  22-Apr-2006  christos Coverity CID 533: Plug memory leak.
Add a new function VarFreeEnv() to free environment variables and use it.
 1.104  31-Mar-2006  christos Add some coverity allocation comments, and change the way the allocator
functions work. When they allocate storage that needs to be freed, instead
of setting a boolean, set the pointer to be freed. Plug some more memory
leaks found by inspection.
 1.103  19-Mar-2006  sjg Fix :P modifier so it actually works as described.
I.e ${var.c:P} should expand to the absolute path of var.c found via .PATH
 1.102  26-Feb-2006  sjg Update man page and add test case for specifying modifiers via variable.
Also allow said variable to appear anywhere in the modifier list.
 1.101  18-Feb-2006  sjg Allow variable modifiers to be specified via variable.
 1.100  27-Aug-2005  christos PR/31077: Wil L: /usr/bin/make can read off of end of buffer
 1.99  08-Aug-2005  christos From Max Okumoto:
- Remove casts to NULL.
- Remove space between cast and object.
 1.98  05-Aug-2005  christos More KNF cleanups from Max Okumoto
 1.97  25-Jul-2005  christos Whitespace KNF cleanup from Max Okumoto
 1.96  01-Jul-2005  christos PR/29985: Roland Illig: make(1) interprets backslash wrongly in for
loop expansions, when the expanded variable ends in backslash and
the backslash is the last character on the line. While this fix is
ugly (detect the condition and append a space), it is the least
intrusive for now.
 1.95  03-Jun-2005  lukem * Improve error handling with unrecognized chars after :t.
* Explicitly goto default_case for unknown chars encountered after
various : modifiers, rather than multiple FALLTHRUs.
* Appease gcc -Wuninitialized for sv_name and v_ctxt.

Discussed with sjg.
 1.94  03-Jun-2005  lukem cosmetic comment tweak
 1.93  01-Jun-2005  sjg Add :Ox for random ordering, based on patch from
Mike M. Volokhov <mishka@apk.od.ua>
 1.92  16-Feb-2005  christos PR/29203, PR/29204: Max Okumoto: KNF changes to make [no functional changes]
 1.91  30-Oct-2004  dsl Add (unsigned char) cast to ctype functions
 1.90  01-Jul-2004  jmc Change to use __unused instead and provide a compat definition in make.h if
not already defined from cdefs.h
 1.89  01-Jul-2004  jmc Add some checks for gcc around a few function declarations and note the
unused variables. Also fix a few other warnings that PR#22118 shows when
trying to compile bmake on non-NetBSD hosts
 1.88  07-May-2004  ross Simplify build, no functional changes.

Instead of adding MAKE_BOOTSTRAP for hosted environments, i.e., when
you want things simple, instead add MAKE_NATIVE to get those hugely
important features like __RCSID().

It's now possible to build make on some hosts with: cc *.c */*.c
 1.87  20-Feb-2004  sjg branches: 1.87.2;
Fix :M so that modifiers in nested variables don't terminate parsing
early. Add a unit-test to verify it works and keeps working.
Re-jig the unit-tests so that all are sub makefiles.
 1.86  24-Jan-2004  dsl Restore correct name string when VarGetPattern() returns.
Stops error from free() evaluating ${x::=y} when x is undefined.
(Erm why is free() allowed to write to fd 2?)
 1.85  08-Jan-2004  sjg Fix :?: modifier so that it works again.
 1.84  26-Dec-2003  jmc Support variable expansions inside of SYSV style = substitutions. Better matches
expectations (and some cases of historical behavior I've found). Also fixes
PR#3865
 1.83  26-Dec-2003  jmc Don't assume startc is always {, set delim to endc when separating the args to
the ? test.
 1.82  23-Oct-2003  jmc Fixes from PR#23210 to eliminate use of asprintf which makes cross building
on non-NetBSD hosts work again
 1.81  27-Sep-2003  sjg Implement :[] modifier to allow picking a range of words out of a variable.
Also :tW and a W flag to :C and :S to allow treating value as a single word.
Add unit tests for the above, and fix some corner cases.
Based on patches supplied by Alan Barrett <apb@cequrux.com>
 1.80  07-Aug-2003  agc Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22365, verified by myself.
 1.79  31-Jul-2003  scw Const poisoning.
 1.78  29-Jul-2003  sjg A couple of other places where delim should be set
 1.77  29-Jul-2003  sjg First bug picked up by the unit tests - delim wasn't always initialized.
Avoid putting '\' in test case script lines since shell's like that on
SunOS insist on interpreting them.
 1.76  28-Jul-2003  sjg Fix parsing bug for :ts - patch from Alan Barrett <apb@cequrux.com>
Also add simple unit-test jig (regress/usr.bin/make will use it too)
but having it local here makes inclusion in bmake simpler.
 1.75  23-Jul-2003  sjg Fix merge problem with ts modifier and const correctness.
Need to pass nstr to VarModify.
 1.74  14-Jul-2003  sjg Add a :ts[c] modifier to allow controlling the separator used between
words in a variable expansion. If 'c' is omitted no separator is used.
 1.73  14-Jul-2003  christos Pass WARNS=3
 1.72  22-May-2003  christos PR/19781: Thomas Klausner: make error message not helpful on unclosed ${var:foo
 1.71  14-Mar-2003  thorpej Add a -X option, which prevents make(1) from putting variables
set on the command line into the environment individually. This
can be useful on systems which have a tight limit on the size
of the argument space.
 1.70  15-Jun-2002  wiz Remove !__STDC__ stuff, de-__P(), ANSIfy, and de-register.
 1.69  21-Mar-2002  christos don't print extra newlines on errors.
 1.68  06-Feb-2002  pk Add `tu' and `tl' variable expansion modifiers, which transform the
value to uppercase and lowercase, respectively. From Kevin Neal from FreeBSD.
 1.67  27-Jan-2002  reinoud Fix major bug in make(1) ... due to shadowing of the dotLast path used for
the .DOTLAST primitive by a boolean variable with the same name, this whole
mechanism was broken ... it doesn't save much stat calls but it was wrong.

Thanks to Jason Thorpe for the other shadow-variable fixing patches he
made.
 1.66  25-Dec-2001  lukem - partially fix ${foo:?true:false} so that at least it now parses ok and
the true result works. for some reason the false result doesn't, even
though make -dv shows it being set as the result. (blah!)
- add braces in comments to fix vi showmatch
 1.65  12-Jun-2001  sjg Add 4th arg (flags) to Var_Set so that VarLoopExpand can tell it not
to export interator variables when using context VAR_CMD.

Reviewed: christos
 1.64  10-Jun-2001  sjg Simplify the exporting of VAR_CMD's via MAKEFLAGS.
We now just list the names of such variables in .MAKEOVERRIDES.
When we come to export MAKEFLAGS we quote the value of each exported variable
using :Q, using: ${.MAKEOVERRIDES:O:u:@v@$v=${$v:Q}@}
The :O:u suppresses duplicate names.
Also modifed Parse_DoVar to re-export MAKEFLAGS whenever .MAKEOVERRIDES
is assigned to so .MAKEOVERRIDES+= PATH will export PATH=${PATH:Q}
to the environment, while .MAKEOVERRIDES= will disable export of VAR_CMD's.
 1.63  09-Jun-2001  sjg Do a better job of duplicate suppression in .MAKEOVERRIDES.
From Var_Set: We actually want the equivalent of
.MAKEOVERRIDES:= ${.MAKEOVERRIDES:Nname=*} name='val'
clearing the previous value for name is important, since
doing simple duplicate suppression does not handle:
$ make FOO=goo
which then runs a sub-make with FOO=boo
the commands from that sub-make should see just FOO=boo.
 1.62  09-Jun-2001  sjg Modify handling of command line variable assignments and their exporting
via MAKEFLAGS. Instead of appending them directly to .MAKEFLAGS, put
them in .MAKEOVERRIDES (and ensure they are quoted). This is now done
in Var_Set when it exports VAR_CMD's.
Use ExportMAKEFLAGS() to export MAKEFLAGS, using the combined content
of .MAKEFLAGS and .MAKEOVERRIDES (with duplicate supression).
If .MAKEFLAGS is assigned to in a Makefile, ExportMAKEFLAGS is called again.
This allows a line like:
.MAKEOVERRIDES=
to effectively stop the exporting of the command line vars in MAKEFLAGS.
 1.61  05-Jun-2001  explorer Perform variable expansion on sysv rhs patterns. That is, expand ${O} in: ${SRCS:.c=.${O}}
 1.60  01-Jun-2001  sjg A number of semi-related changes.
1. make -dx turns on DEBUG_SHELL which causes sh -x to be used where
possible.
2. PrintOnError() is now called when make is stopping due to an error.
This routine reports the curdir and the value of any variables listed
in MAKE_PRINT_VAR_ON_ERROR.
3. Variables set via command line, are propagated to child-makes via
MAKEFLAGS. This behaviour appears to be necessary for POSIX (according
to the GNU folk anyway).
4. Do not reset MAKEFILE when reading ".depend" as this rather eliminates the
usefulness of ${MAKEFILE}.
5. Added ${.newline} as a simple means of being able to include \n in the
result of a :@ loop expansion.
6. Set ${MAKE_VERSION} if defined. Need to come up with a useful value.

Reviewed: christos
 1.59  12-May-2001  sjg Modifier :P avoid segfault if gn is found but path is NULL.
 1.58  10-Mar-2001  itojun redo of previous: do not destroy v->val if str points to v->val.
XXX GC of v->val
 1.57  09-Mar-2001  itojun correct case with VAR_KEEP. if clause checked VAR_KEEP backwards,
and made str a dangling pointer.
 1.56  05-Sep-2000  christos fixed comment U -> u
 1.55  05-Sep-2000  christos :u modifier a'la uniq(1) [from der Mouse]
 1.54  13-Aug-2000  christos don't attempt to free varNoError too.
 1.53  10-Jun-2000  sjg branches: 1.53.2;
no need for v->name-- on JUNK vars anymore
 1.52  10-Jun-2000  mycroft For VAR_JUNK variables, defer freeing of the name string until the bottom of
Var_Parse(), as it may be used (strdup()ed) by some modifiers.
 1.51  10-Jun-2000  sjg Don't free v until after the last use.
 1.50  06-Jun-2000  mycroft Do some evil with VAR_KEEP to make it happier. This seems... wrong.
 1.49  06-Jun-2000  mycroft Allow further patterns after a :D or :U, as in ODE make. (Copied some code
from VarGetPattern. It should become more flexible instead.)
 1.48  06-Jun-2000  mycroft Roll back my for-expansion changes. Doing this right is just too hard to be
worth it.
 1.47  01-Jun-2000  sjg Make ::= work again with the VAR_FOR context changes.
 1.46  01-Jun-2000  mycroft Create a `for' context, and substitute iteration variable from it using the
normal Var_Parse() path. This allows :R, etc. to work on iteration variables.
 1.45  01-Jun-2000  sjg Rats! ${FOO:=bar} is a common usage of the SysV = modifier.
To avoid that, we now do ::[+?!]*= but the SysV = modifier can
conflict with any new modifier. At there are currently no Makefiles
in our tree that use ${FOO::=bar}
 1.44  30-May-2000  sjg Implemented a set of assignment modifiers. These solve obscure
problems such as using modifiers on .for loop iterators derived from
local variables (eg .TARGET).
Unless the variable already exists in a global context, these assignments are
local to the current context (this is usually what is wanted).
 1.43  14-May-2000  sjg branches: 1.43.2;
Expand variables that appear in a [No]Match pattern. Allows
[ x${DONE:M${.TARGET}} != x ] || blah
 1.42  11-May-2000  sjg Expand variables in variable name passed to Var_Set and Var_Append.
This allows ${FOO}.bar=fubar etc to work.
This does not impact any Makefiles in the current tree but allows
some cool magic (which I need elsewhere).
 1.41  11-May-2000  sjg Pass ctxt along to VarLoopExpand so that local variables such as .TARGET
can be expanded.
 1.40  29-Apr-2000  sjg Added the :U :D :L :P :! and :@ modifiers as used by ODE make.
These allow some very useful magic in the makefiles.
The comment in var.c describing their behaviour is mostly lifted
from ODE make, but the implementation of the modifiers themselves
is quite different (much simpler) due to divergence of our code base.
 1.39  17-Apr-2000  christos Don't type && when you mean || (pointed out by David Holland)
 1.38  16-Apr-2000  christos PR/9898: David Holland: small glitch in var substitution for single letter
variables.
 1.37  15-Sep-1999  sommerfeld Reuse the Hash_Entry `name' field to store the variable name when we
can (i.e., everything except environment variables, which aren't
stored in hash tables).

While we're here, inline the body of VarDelete into Var_Delete since
it's the only caller and it's just simpler that way when v->name can
share storage with the hash entry and may not need to be freed
separately.

Speeds up the infamous libc build benchhmark maybe 1% on PIII, 4% on
alpha pc164

Suggested by Perry Metzger.
 1.36  15-Sep-1999  mycroft Replace the linear variable list in the GNode with a hash table. Gives a
small (~10% on my machine) speedup building libc.
Changes by: perry (+ bug fix and minor change by me)
Reviewed by: mycroft sommerfeld
 1.35  12-Sep-1999  christos Add :O var modifier, that sorts the words in a variable.
 1.34  06-Jun-1999  christos Revert previous change. I am a bonehead.
 1.33  06-Jun-1999  christos remove accidentally left debugging code
 1.32  06-Jun-1999  christos Allow junk variable to work in non dynamic contexts. Allows as to say things
like:
.if defined(COPTS) && !empty(COPTS:M*-g*)
 1.31  03-Apr-1999  gwr Make regex support conditional on NO_REGEX (was MAKE_BOOTSTRAP)
Solaris has a compatible regex, as do many others now...
 1.30  06-Nov-1998  christos - full prototypes
- more conservative printf
- char -> unsigned char
 1.29  01-Nov-1998  itohy Reduced memory leaks.
I found some more leaks, but are not in inside of iterations.
 1.28  13-Oct-1998  wsanchez Include sys/types.h
 1.27  18-Sep-1998  christos Fix conditional variable expression parsing (reported by cgd)
 1.26  03-Apr-1998  cgd in Var_Parse(), set delim in a couple of other paths into cleanup
 1.25  01-Apr-1998  christos Add conditional variable evaluation from Alistair Crooks.
 1.24  26-Mar-1998  christos PR/5210: Hauke Fath: make core dumps with .SHELL
Unfortunately this revealed a deeper problem with the brk_string code.
To fix it:
- remove sharing of the buffer between brk_string invocations
- change the semantics of brk_string so that the argument array
starts with 0, and return the buffer where the strings are
stored
 1.23  28-Sep-1997  lukem branches: 1.23.2;
wrap #include <sys/cdefs.h>, __RCSID(...) stuff in #ifndef MAKE_BOOTSTRAP
 1.22  22-Sep-1997  christos Expand variables within variables correctly; eg.
foo.123=abc
bar=123
all:
echo ${foo.${bar}}
 1.21  10-Jul-1997  christos Fix memory corruption that was caused by debugging code leftovers.
 1.20  07-Jul-1997  christos PR/3825: David A. Holland: global regexp handling gets stuck in infinite loop.
Actually there were two bugs:
- Add REG_NOTBOL after the first substitution.
- Handle the rm_so == rm_eo == 0 case, where in a substitution such
as 's/bzzzt/z*/g' the first time z* matches nothing.
 1.19  01-Jul-1997  christos Add WARNS=1
RCSID police
 1.18  18-Mar-1997  christos Fix problems in the RE substitution error handling.
 1.17  31-Dec-1996  christos Don't compile the regex code if MAKE_BOOTSTRAP (from gwr)
 1.16  24-Dec-1996  christos [initial version of the substitution/regexp changes were courtesy of Der Mouse]

- fix the variable substitution code in make [PR/2748]
1. change s/a/b/ so that it substitutes the first occurance of the
pattern on each word, not only the first word.
2. add flag '1' to the variable substitution so that the substitutions
get performed only once.

***THIS IS AN INCOMPATIBLE CHANGE!***

Unfortunately there was no way to make things consistent without
modifying the current behavior. Fortunately none of our Makefiles
depended on this.

OLD:

VAR = aa1 aa2 aa3 aa4

S/a/b/ = ba1 aa2 aa3 aa4
S/a/b/g = bb1 bb2 bb3 bb4

NEW:

VAR = aa1 aa2 aa3 aa4

S/a/b/ = ba1 ba2 ba3 ba4
S/a/b/1 = ba1 aa2 aa3 aa4
S/a/b/g = bb1 bb2 bb3 bb4
S/a/b/1g = bb1 aa2 aa3 aa4

- add regexp variable substitution via 'C/foo/bar/' [PR/2752]

- add variable quoting via the ${VAR:Q} modifier. This is useful when running
recursive invocations of make(1):

make VAR=${VAR:Q}

will always work... (This may prove useful in the kernel builds...) [PR/2981]
 1.15  06-Nov-1996  christos - Merge in FreeBSD and Lite2 changes.
- Fix bug where a non-archive target with a .a suffix would always
be considered to be out of date, since it does not have a TOC.
 1.14  13-Aug-1996  christos Add estrdup(), a checked version of strdup and use it.
 1.13  28-May-1996  christos - Move -D flags from Makefile to config.h and explain what they do. Add
-Wall -Wno-unused to CFLAGS. Add new define SYSVVARSUB to enable SysV
style variable substitutions and enable them.
- Add SunOS style command substitutions via SUNSHCMD
- Fix core dump with '{variable = value'
 1.12  02-Nov-1995  christos branches: 1.12.4;
Minor:
- ${.PREFIX} should never contain a full pathname
- Fixed gcc -Wall warnings
Major:
- compatMake is now FALSE. This means that we are now running in
full pmake mode:
* rules on dependency lines can be executed in parallel and or
out of sequence:

foo: bar baz

can fire the rule for baz before the rule for bar is fired.
To enforce bar to be fired before baz, another rule needs to be
added. [bar: baz]
* adjacent shell commands in a target are now executed by a single
invocation of the shell, not one invocation of the shell per line
(compatMake can be turned off using the -B flag)
- The -j flag now works... I.e. make -j 4 will fork up to four jobs in
parallel when it can. The target name is printed before each burst
of output caused by the target execution as '--- target ---', when j > 1
- I have changed all the Makefiles so that they work with make -j N, and
I have tested the whole netbsd by:
'make -j 4 cleandir; make -j 4 depend; make -j 4; make -j 4 install'
- I have not compiled or tested this version of make with -DREMOTE.
 1.11  14-Jun-1995  christos - $NetBSD$ rcsids
- Fixed so that .[A-Z]* targets that do not match keywords are ignored as
Posix mandates
- Added .PHONY target keyword
 1.10  04-Feb-1995  christos str.c: Don't free NULL; does not work with non posix frees.
var.c: Minor memory leak plugged.
suff.c: Don't add extra sources on the null suffix if it has dependency
lines or commands attached to it [POSIX says so]
 1.9  20-Jan-1995  christos Fixed to parse correctly SysV variable substitutions ${VAR:P1=P2} when
P1 or P2 contain }'s or )'s. From mikeb@snow.datametrics.com
 1.8  09-Jan-1995  christos Save the name of the environment variables. Remove the comment that
says that we don't. from Paul Borman (prb@bsdi.com)
 1.7  16-Jun-1994  jtc Christos' fix for quoting variable extraction properly.
 1.6  06-Jun-1994  jtc Fixes from Christos Zoulas, who used purify, objectcenter and testcenter
to find memory leaks and illegal memory accesses.
 1.5  05-Mar-1994  cgd fixes/improvements from Christos Zoulas <christos@deshaw.com>.
 1.4  13-Jan-1994  jtc Include appropriate header files to bring prototypes into scope.
 1.3  13-Jan-1994  cgd add missing arg to setenv(). reported by downsj@csos.orst.edu
 1.2  01-Aug-1993  mycroft Add RCS identifiers.
 1.1  21-Mar-1993  cgd branches: 1.1.1;
Initial revision
 1.1.1.2  28-Dec-1996  tls Import 4.4BSD-Lite2 sources onto CSRG branch (already merged at head)
 1.1.1.1  21-Mar-1993  cgd initial import of 386bsd-0.1 sources
 1.12.4.1  26-Jan-1997  rat Update make(1) from trunk, by request from Christos Zoulas. Fixes many bugs.
 1.23.2.1  08-May-1998  mycroft Sync with trunk, per request of christos.
 1.43.2.1  23-Jun-2000  minoura Sync w/ netbsd-1-5-base.
 1.53.2.1  07-Jun-2001  he Pull up revision 1.61 (requested by explorer):
Expand variables in SysV patterns, as in ``${SRCS:.c=.${O}}''.
 1.87.2.1  10-May-2004  tron Pull up revision 1.88 (requested by sjg in ticket #282):
Simplify build, no functional changes.
Instead of adding MAKE_BOOTSTRAP for hosted environments, i.e., when
you want things simple, instead add MAKE_NATIVE to get those hugely
important features like __RCSID().
It's now possible to build make on some hosts with: cc *.c */*.c
 1.110.2.2  19-May-2006  christos Coverity CI D3758: Plug memory leak.
 1.110.2.1  19-May-2006  christos file var.c was added on branch chap-midi on 2006-05-19 17:29:02 +0000
 1.115.4.1  03-Sep-2007  wrstuden Sync w/ NetBSD-4-RC_1
 1.115.2.1  05-Jun-2007  bouyer Apply patch (requested by tron in ticket #696):
usr.bin/make/compat.c patch
usr.bin/make/cond.c patch
usr.bin/make/dir.c patch
usr.bin/make/for.c patch
usr.bin/make/main.c patch
usr.bin/make/make.1 patch
usr.bin/make/make.c patch
usr.bin/make/make.h patch
usr.bin/make/nonints.h patch
usr.bin/make/parse.c patch
usr.bin/make/str.c patch
usr.bin/make/targ.c patch
usr.bin/make/util.c patch
usr.bin/make/var.c patch

Synchronize make(1) with HEAD branch to increase perfomance and
improve stability.
 1.117.4.1  06-Nov-2007  matt sync with HEAD
 1.128.4.2  18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.128.4.1  23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.128.2.1  18-May-2008  yamt sync with head.
 1.149.2.1  13-May-2009  jym Sync with HEAD.

Third (and last) commit. See http://mail-index.netbsd.org/source-changes/2009/05/13/msg221222.html
 1.166.2.1  23-Jun-2011  cherry Catchup with rmind-uvmplock merge.
 1.167.2.4  22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.167.2.3  16-Jan-2013  yamt sync with (a bit old) head
 1.167.2.2  30-Oct-2012  yamt sync with head
 1.167.2.1  23-May-2012  yamt sync with head.
 1.171.2.3  20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.171.2.2  23-Jun-2013  tls resync from head
 1.171.2.1  20-Nov-2012  tls Resync to 2012-11-19 00:00:00 UTC
 1.184.2.1  10-Aug-2014  tls Rebase.
 1.208.2.2  26-Apr-2017  pgoyette Sync with HEAD
 1.208.2.1  20-Mar-2017  pgoyette Sync with HEAD
 1.209.2.1  21-Apr-2017  bouyer Sync with HEAD
 1.215.4.2  09-Jun-2018  martin Pull up following revision(s) (requested by christos in ticket #870):

sys/lib/libgnuefi/Makefile.inc: revision 1.5
external/mit/xorg/xorg-pkg-ver.mk: revision 1.9
sys/arch/i386/stand/lib/Makefile.inc: revision 1.17
sys/compat/common/Makefile.inc: revision 1.25
usr.bin/make/unit-tests/varquote.exp: revision 1.1, 1.2
sys/lib/libsa/Makefile.inc: revision 1.24
usr.bin/make/var.c: revision 1.220
sys/arch/ia64/stand/efi/libefi/Makefile.inc: revision 1.5
sys/arch/hppa/spmath/Makefile.inc: revision 1.10
usr.bin/make/unit-tests/varquote.mk: revision 1.1, 1.2
usr.bin/make/unit-tests/Makefile: revision 1.53
sys/lib/libkern/Makefile.inc: revision 1.45
sys/arch/acorn32/stand/lib/Makefile.inc: revision 1.4
sys/arch/m68k/fpsp/Makefile.inc: revision 1.21
usr.bin/make/make.1: revision 1.273
sys/lib/libz/Makefile.inc: revision 1.17

- Introduce :q modifier for make variables and make it double escape $'s so
that passing variables to recursive makes with :q works as expected.
- Adjust makefiles that use recursive make to use :q

Discussed on tech-toolchain@

XXX: pullup 8
 1.215.4.1  07-Jun-2018  martin Pull up following revision(s) (requested by christos in ticket #834):

usr.bin/make/var.c: revision 1.219
share/mk/bsd.obj.mk: revision 1.52
usr.sbin/crash/Makefile: revision 1.40
share/mk/sys.mk: revision 1.131
share/mk/bsd.sys.mk: revision 1.276
share/mk/bsd.sys.mk: revision 1.277

Prevent 'make obj' creating an objdir inside an objdir. This happens the
second time we call 'make obj' in /usr/src/rescue with MAKEOBJDIR set.
XXX: pullup-8
-
Need to delete ${DESTDIR} for lint
XXX: pullup-8
-
Fix reproducible builds
XXX: pullup-8
-
Also substitute ${X11SRCDIR} for the random seed for reproducible builds so
that the random seed is consistent.
XXX: pullup-8
-
Since ${MAKE} converts $$ -> $ during parsing we need to put it back to
preserve the original variable value with :Q.
XXX: pullup-8
-
Be more selective about remapping object directories otherwise we pick
objdump and objcopy with the regex.
XXX: pullup-8
 1.218.2.2  26-Dec-2018  pgoyette Sync with HEAD, resolve a few conflicts
 1.218.2.1  25-Jun-2018  pgoyette Sync with HEAD
 1.220.2.1  10-Jun-2019  christos Sync with HEAD
 1.930.2.1  31-May-2021  cjep sync with head
 1.1121.2.2  02-Aug-2025  perseant Sync with HEAD
 1.1121.2.1  01-Jul-2024  perseant Sync with HEAD.

RSS XML Feed