Home | History | Annotate | Download | only in make
History log of /src/usr.bin/make/dir.c
RevisionDateAuthorComments
 1.297  12-Jun-2025  rillig make: use a common style for unexpected error messages

In enomem, report the actual error instead of a fixed ENOMEM.
 1.296  11-Apr-2025  rillig make: convert more messages to the "file:123" format
 1.295  07-Jul-2024  rillig make: only generate code for cleanup functions in CLEANUP mode
 1.294  31-May-2024  rillig make: clean up API for iterating over hash tables
 1.293  25-May-2024  rillig make: fix some more memory leaks
 1.292  25-May-2024  rillig make: fix a few more memory leaks
 1.291  24-May-2024  rillig make: remove trailing whitespace
 1.290  20-May-2024  sjg make: cleanup style in FindFile

Since we removed the last check for isinclude,
restore to the previous indentation etc.

No functional change.
 1.289  20-May-2024  sjg Allow .include <> to search .CURDIR last

libc at least appears to depend on this.
 1.288  19-May-2024  sjg make: use separate function to include makefiles.

Have Dir_FindFile and Dir_FindInclude call FindFile with a
bool flag to indicate whether .CURDIR should be be searched at all.
 1.287  19-May-2024  sjg Dir_FindFile treat sysIncPath and defSysIncPath as special

Rather than play games with adding .DOTLAST and then having to
exclude that from .SYSPATH, just have Dir_FindFile check if
path is sysIncPath or defSysIncPath and behave as if .DOTLAST seen
and also skip the final search of .CURDIR.

Fix Dir_SetSYSPATH to use defSysIncPath if sysIncPath is empty.
 1.286  29-Dec-2023  rillig make: fix declared types of list nodes

No functional change.
 1.285  19-Dec-2023  rillig make: clean up comments

No binary change, except for line numbers in assertions.
 1.284  17-Dec-2023  rillig make: clean up names of local variables

No binary change.
 1.283  21-Sep-2023  rillig make: remove obsolete comments, clean up comments and identifiers

No binary change, except for the line numbers in assertions.
 1.282  23-Jun-2023  rillig make: warn about malformed patterns in ':M', ':N' and '.if make(...)'

These patterns shouldn't occur in practice, as their results are tricky
to predict. Generate a warning for now, and maybe an error later.

Reviewed by sjg@.
 1.281  22-Jun-2023  rillig make: clean up comments related to pattern matching
 1.280  24-Jan-2023  sjg make: .SYSPATH: to add dirs to sysIncPath

.SYSPATH: with no sources will clear sysIncPath
otherwise sources are added

Reviewed by: rillig
 1.279  07-May-2022  rillig make: fix grammar in comment of DirFindDot
 1.278  04-Feb-2022  rillig make: use unsigned int for line numbers everywhere

Previously, some line numbers were stored as signed int while others
were stored as size_t. Since line numbers are never negative, use an
unsigned type. Since the maximum file size for makefiles is 1 GB (see
loadfile), unsigned int is large enough even on 64-bit platforms.

Using a single data types reduces the number of type conversions. Using
unsigned int improves compatibility with C90 (printf %u instead of %zu),
which is needed by bmake, which is derived from usr.bin/make.

No functional change.
 1.277  30-Jan-2022  christos Make the GNode lineno unsigned to fix lint warning in var.c calling
PrintLocation()
 1.276  15-Dec-2021  rillig make: use consistent indentation for statements and continuations

No binary change, except for line numbers in assertions in suff.c.
 1.275  28-Nov-2021  rillig make: eliminate CachedStatsFlags

Having two boolean flags as parameters should be easier to understand
than bit manipulations. The variable names now match more directly.

No functional change.
 1.274  28-Nov-2021  rillig make: convert GNodeFlags from enum into bit-fields

Now that Enum_ToString is implemented for each type separately, it's
easy to convert them to bit-fields. This gets rid of the magic numbers
12 for CYCLE and 13 for DONECYCLE that left a suspicious gap in the
numbers. This gap was not needed since the code didn't make use of the
relative ordering of the enum constants.

The effects of this conversion are fewer capital letters in the code,
smaller scope for the GNode flags, and clearer code especially when
setting a flag back to false.

One strange thing is that GCC 10.3.0 doesn't optimize GNodeFlags_IsNone
to an single bitmasking instruction, at least on x86_64. Instead it
generates a testb instruction for each of the flags, even loading bit 8
separately from the others. Clang 12.0.1 knows this optimization
though and generates the obvious sequence of movzwl, testl, jz.

No functional change.
 1.273  21-Sep-2021  rillig make: remove unnecessary const from parameters

These were leftovers from earlier refactorings, when extracting code to
separate functions.

No functional change.
 1.272  04-Apr-2021  rillig make: remove filler word 'Do' from function names for parsing

No functional change, except for debug logging.
 1.271  03-Apr-2021  rillig make: use C99 bool type instead of defining its own

No functional change.
 1.270  05-Feb-2021  rillig make: add const to SearchPath_Print
 1.269  05-Feb-2021  rillig make: add shortcut Global_Delete for deleting a global variable
 1.268  04-Feb-2021  rillig make: rename some VAR constants to SCOPE

The word "context" does not fit perfectly to the variables that are
associate with a GNode, as the context is usually something from the
outside and the variables are more like properties inherent to the
GNode.

The term "global context" fits even less. Since the thing where
variables are looked up is commonly named a scope, use that term
instead.

This commit only renames the global variables VAR_GLOBAL, VAR_INTERNAL
and VAR_CMDLINE, plus a few very closely related comments. These are:

GNode.vars (because of line breaks)
GNode_Free (dito)
varname-make_print_var_on_error.mk
varname-make_print_var_on_error-jobs.mk

The debug message in Var_Stats is left as-is since there is no unit test
for it yet.

The other renamings (variable names "context", "ctxt", as well as
further comments) will be done in a follow-up commit.
 1.267  03-Feb-2021  rillig make: replace Global_AppendExpand with Global_Append

All callers with a variable name that is guaranteed to not contain a
dollar sign have been converted to call Global_Append instead of the
previous Global_AppendExpand. After that, Global_AppendExpand was
unused, therefore it was effectively just renamed.
 1.266  03-Feb-2021  rillig make: use shortcut functions Global_SetExpand and Global_AppendExpand

There are many places where global variables are set or appended to. To
reduce clutter and code size, encode the VAR_GLOBAL in the function
name.

The word Expand in the function names says that the variable name is
expanded. In most of the cases, this is not necessary, but there are no
corresponding functions Global_Set or Global_Append yet.

Encoding the information whether the name is expanded or not in the
function name will make inconsistencies obvious in future manual code
reviews. Letting the compiler check this by using different types for
unexpanded and expanded variable names is probably not worth the effort.
There are still a few bugs to be fixed, such as in SetVar, which expands
the variable name twice in a row.
 1.265  30-Jan-2021  rillig make(1): split Buf_Destroy into Buf_Done and Buf_DoneData

In all cases except one, the boolean argument to Buf_Destroy was
constant. Removing that argument by splitting the function into two
separate functions makes the intention clearer on the call site. It
also removes the possibility for using the return value of Buf_Done,
which would have made no sense.

The function Buf_Done now pairs with Buf_Init, just as in HashTable and
Lst.

Even though Buf_Done is essentially a no-op, it is kept as a function,
both for symmetry with Buf_Init and for clearing the Buffer members
after use (this will be done only in CLEANUP mode, in a follow-up
commit).
 1.264  24-Jan-2021  rillig make(1): convert SearchPath to struct

This prepares for making dotLast a simple struct member instead of a
fake CachedDir, which is easier to understand.
 1.263  23-Jan-2021  rillig make(1): split Dir_FindFile into separate functions
 1.262  23-Jan-2021  rillig make(1): remove the remaining beasts from the comments
 1.261  23-Jan-2021  rillig make(1): extend comments in dir.c
 1.260  23-Jan-2021  rillig make(1): rename Dir_AddDir, reorder parameters of SearchPath_ToFlags
 1.259  23-Jan-2021  rillig make(1): extract SearchPath_ExpandMiddle from SearchPath_Expand
 1.258  23-Jan-2021  rillig make(1): split local variable in SearchPath_Expand
 1.257  23-Jan-2021  rillig make(1): rename parameter of SearchPath_Expand
 1.256  23-Jan-2021  rillig make(1): rename Dir_Expand to SearchPath_Expand

The main subject of this function is the search path. In this search
path the pattern is expanded.
 1.255  10-Jan-2021  rillig make(1): consistently use boolean expressions in conditions

Most of the make code already followed the style of explicitly writing
(ptr != NULL) instead of the shorter (ptr) in conditions.

The remaining 50 instances have been found by an experimental,
unpublished check in lint(1) that treats bool expressions as
incompatible to any other scalar type, just as in Java, C#, Pascal and
several other languages.

The only unsafe operation on Boolean that is left over is (flags &
FLAG), for an enum implementing a bit set. If Boolean is an ordinary
integer type (the default), some high bits may get lost. But if Boolean
is the same as _Bool (by compiling with -DUSE_C99_BOOLEAN), C99 6.3.1.2
defines that a conversion from any scalar to the type _Bool acts as a
comparison to 0, which cannot lose any bits.
 1.254  30-Dec-2020  rillig make(1): format multi-line comments
 1.253  27-Dec-2020  rillig make(1): exit 2 on technical errors

This allows the -q option to distinguish errors from out-of-date
targets. Granted, it's an edge case but it should be solved
consistently anyway.

The majority of cases in which make exits with exit status 1, even in -q
mode, is when there are parse errors. These have been kept as-is for
now as they affect many of the unit tests.

The technical errors, on the other hand, occur so rarely that it's hard
to write reliable tests for them that fail consistently on all platforms
supported by make.
 1.252  13-Dec-2020  rillig make(1): add str_basename to reduce duplicate code

The function basename from POSIX has a few unfortunate properties, it is
allowed to return a pointer to static memory. This is too unreliable,
therefore this trivial own implementation.
 1.251  06-Dec-2020  rillig make(1): remove comment decoration
 1.250  06-Dec-2020  rillig make(1): inline macros for debug logging

No changes to the resulting binary, except for the line numbers in
assertions.
 1.249  04-Dec-2020  rillig make(1): use consistent variable names for list nodes
 1.248  04-Dec-2020  rillig make(1): use fixed format for debug output of the directory cache

The previous output format had a %-20s conversion specifier. This
produced different output depending on the length of the pathname, which
was too difficult to normalize. By moving the directory name to the
end, it is no longer necessary to fill up any space, and the numbers are
always aligned properly.

As a result, 3 of the unit tests no longer need any special
postprocessing of their output.
 1.247  01-Dec-2020  rillig make(1): rename local variable in Dir_SetPATH

The variable name should reflect the close relationship to the .DOTLAST
keyword that can be used in search paths.
 1.246  01-Dec-2020  rillig make(1): remove Dir_InitDir

The function name had been too ambiguous since it didn't mention the
particular directory that was initialized. Instead of that function,
Dir_InitCur is called directly from main_Init.

The pseudo CachedDir entry ".DOTLAST" is initialized at the very
beginning. The observable behavior is unchanged since this a
memory-only object with no connection to the file system.
 1.245  30-Nov-2020  rillig make(1): clean up variable names in Dir_FindFile

The special path entry is called .DOTLAST, therefore the local variable
should have the same name.

A variable named 'base' must not point to the slash of a pathname. It
may only point to the character after the slash, everything else is
confusing, even if it's only for a brief moment.
 1.244  30-Nov-2020  rillig make(1): document difference between 'cur' and 'dot'
 1.243  30-Nov-2020  rillig make(1): fix memory leak for lstat cache in -DCLEANUP mode
 1.242  29-Nov-2020  rillig make(1): initialize global variables in dir.c

Calling CachedDir_Assign requires that the variable be initialized. On
most systems, NULL is represented as all-zero bits already. This change
is only for the few other systems.

Add some comments explaining the implementation of Dir_AddDir since that
is tricky to read from the code alone.
 1.241  29-Nov-2020  rillig make(1): clean up memory management for CachedDirs

Previously, the reference count for a newly created CacheDir had been
set to 1 in CacheNewDir. This was wrong because at that point, the
object had not been referenced by any nonlocal variable. The reference
count is no longer incremented at this point.

All callers of CacheNewDir either append the newly created CachedDir to
a SearchPath via Lst_Append and CachedDir_Ref, or they assign it to a
global variable via CachedDir_Assign.

Since the reference count is no longer wrongly incremented, it does not
need to be decremented more than necessary in Dir_End. To keep the code
simple and maintainable, all assignments to global variables are now
handled by CachedDir_Assign. Adding a CachedDir to a list is still done
manually via Lst_Append, and the corresponding code for decrementing is
in SearchPath_Clean and SearchPath_Free. These details may be cleaned
up in a follow-up commit.

As a result, when OpenDirs_Done is called in the unit tests, the list of
open directories is empty. It had been non-empty in a single unit test
before (dep-wildcards.mk), as a result of calling Dir_Expand.

The additional debug logging for the reference counting is not enabled
by default since it contains memory addresses, which makes the output
dependent on the memory allocator.

The function CachedDir_Destroy has been merged into CachedDir_Undef,
which had only been used in Dir_End before. The new name emphasizes
that it corresponds to CachedDir_Ref.
 1.240  29-Nov-2020  rillig make(1): fix the reference count of dotLast going negative

The memory management for dotLast is quite simple. It is initialized
exactly once main_Init > Init_Objdir > Dir_InitDir and freed exactly
once in main_CleanUp > Dir_End. Previously, dotLast was not freed at all.

The first call to CachedDir_Unref decremented the refCount to 0 but
didn't free anything. Next, CachedDir_Destroy was called, which
decremented the reference count to -1, therefore skipping the actual
freeing. This was probably an implementation mistake.

Since Dir_End is called at the very end of main_CleanUp, no code
accesses dotLast after it has been freed.
 1.239  29-Nov-2020  rillig make(1): move CachedDir_Destroy up to the related functions
 1.238  29-Nov-2020  rillig make(1): extract CachedDir_Free0 from CachedDir_Destroy
 1.237  29-Nov-2020  rillig make(1): remove wrong comment in Dir_InitCur

In a makefile with repeated ".CURDIR=." lines, Dir_AddDir is called with
a NULL path, once per line. Since the path is NULL, the search for
OpenDirs_Find is skipped and the directory is always read from disk.
The freshly read directory has a refCount of 1, and the refCount never
raises above 2.

In Dir_InitCur, the directory of the previous .CURDIR has a refCount of
2, which is decremented twice and then freed. After this, the new
directory is placed in the global 'cur', after incrementing its refCount
to 2.

It still seems wrong that the refCount of 'cur' is 2 instead of 1, but
it works well.
 1.236  29-Nov-2020  rillig make(1): add debug logging for OpenDirs_Done
 1.235  29-Nov-2020  rillig make(1): extract CacheNewDir from Dir_AddDir

Change the debug output for directories that are not found.
 1.234  29-Nov-2020  rillig make(1): make documentation of CachedDir.refCount more precise
 1.233  29-Nov-2020  rillig make(1): add debug logging for reference counting of CachedDir
 1.232  29-Nov-2020  rillig make(1): normalize order of declarations in dir.c
 1.231  29-Nov-2020  rillig make(1): remove comment about returning const

The callers modify the reference count of the CachedDir, therefore it
must be modifiable.
 1.230  29-Nov-2020  rillig make(1): rename Dir_Destroy to CachedDir_Destroy

It is no longer exported by the Dir module.
 1.229  29-Nov-2020  rillig make(1): unexport CachedDir internals
 1.228  29-Nov-2020  rillig make(1): reduce memory allocation for dirSearchPath
 1.227  28-Nov-2020  rillig make(1): reduce memory allocation in OpenDirs
 1.226  28-Nov-2020  rillig make(1): replace void pointer in Dir_Destroy with proper pointer
 1.225  28-Nov-2020  rillig make(1): replace Dir_Destroy with SearchPath_Free

The function Dir_Destroy is an implementation detail of the cached
directories, and it should not be exported to the other modules. The
search paths, on the other hand, are the high-level API that may be used
by the other modules, as the concept of search paths is documented in
the manual page.
 1.224  28-Nov-2020  rillig make(1): rename some Dir functions to SearchPath

These functions have the search path as their main subject.
 1.223  28-Nov-2020  rillig make(1): reduce pointer indirection for GNode.implicitParents
 1.222  23-Nov-2020  rillig make(1): use comparisons in boolean expressions

The generated code stays exactly the same.
 1.221  23-Nov-2020  rillig make(1): indent remaining functions in dir.c with tabs
 1.220  23-Nov-2020  rillig make(1): extract ResolveMovedDepends from ResolveFullName
 1.219  23-Nov-2020  rillig make(1): extract ResolveFullName from Dir_UpdateMTime
 1.218  23-Nov-2020  rillig make(1): in Dir_Expand, don't re-use local variables

While here, add a few remarks from a previous attempt at flattening
the function.
 1.217  23-Nov-2020  rillig make(1): update the deactivated code in Dir_FindFile

While here, reduce the scope of the variable ln.
 1.216  23-Nov-2020  rillig make(1): indent Dir_Expand with tabs instead of spaces
 1.215  23-Nov-2020  rillig make(1): flatten Dir_Expand

While here, leave comments in all places where unexpected edge cases
might have hidden.
 1.214  23-Nov-2020  rillig make(1): use properly typed comparisons in boolean contexts
 1.213  23-Nov-2020  rillig make(1): align end-of-line comments with tabs
 1.212  23-Nov-2020  rillig make(1): indent dir.c using tabs instead of spaces

Except for Dir_Expand and Dir_UpdateMTime, which are nested too deeply
to use tabs right now. They first have to be split into separate
functions.
 1.211  23-Nov-2020  rillig make(1): migrate CachedDir.files from HashTable to HashSet
 1.210  14-Nov-2020  rillig make(1): replace a few HashTable_CreateEntry with HashTable_Set

Instead of HashTable_CreateEntry and HashEntry_Set, several places just
need the HashEntry for storing a value in it. This makes the calling
code simpler to understand.

These parts of the code are already hard enough to understand since they
are about memory management and aliasing. Having a too detailed API for
the HashTable only distracts from these topics.
 1.209  14-Nov-2020  rillig make(1): inline local variable in Dir_UpdateMTime

Now that the signature of cached_stats is cleaned up, the line is short
enough to contain the whole condition.
 1.208  14-Nov-2020  rillig make(1): remove redundant struct make_stat

In the cache for stat(2) and lstat(2), only one of the two timestamps
was ever used. To prevent a result from stat(2) leaking into the cache
for lstat(2), there have been two completely separate caches all the
time. Using different fields in the struct was therefore unnecessary.

By removing the redundant field, the internal struct in the cache is the
same as the external struct. This makes one of them redundant, thus
struct make_stat has been renamed to cached_stat, which better describes
its purpose, and the internal struct cache_st has been removed.

Just as before, the cache prevents any direct access to its internal
data. When passing it to the caller, it is copied.

Just as before, the field names of struct cached_stat cannot correspond
to those from struct stat, since the latter are often defined as macros.
Therefore they are prefixed with cst instead of st.

The redundancy had been added on 2020-06-05.
 1.207  14-Nov-2020  rillig make(1): remove redundant parameter from cached_stats

The hash table for the cached data depends only on the passed flags,
therefore the caller does not need to know about their existence.
 1.206  14-Nov-2020  rillig make(1): clean up cached_stats

No functional change.
 1.205  14-Nov-2020  rillig make(1): clarify what 'recently' means in the comments in dir.c
 1.204  14-Nov-2020  rillig make(1): fix typo in comments

This typo may have been influenced by all the '$' in the code.
 1.203  14-Nov-2020  rillig make(1): flatten Dir_InitCur
 1.202  08-Nov-2020  rillig make(1): document inefficient cache lookup in DirMatchFiles
 1.201  08-Nov-2020  rillig make(1): change return type of Dir_MTime to void

Only some callers actually needed the updated time, and because of the
many branches, it was difficult to see that the return value was indeed
gn->mtime all the time.
 1.200  08-Nov-2020  rillig make(1): change return type of Arch_MTime to void

This makes it easier to prove that Dir_MTime always returns gn->mtime,
without looking at the implementation of Arch_UpdateMTime.
 1.199  08-Nov-2020  rillig make(1): change return type of Arch_MemberMTime to void

This makes it easier to prove that Dir_MTime always returns gn->mtime,
without looking at the implementation of Arch_UpdateMemberMTime.
 1.198  07-Nov-2020  rillig make(1): use proper enum constant instead of 0 for CachedStatsFlags
 1.197  07-Nov-2020  rillig make(1): make API of Buf_Init simpler

In most cases, the caller doesn't want to specify the exact number of
preallocated bytes.
 1.196  07-Nov-2020  rillig make(1): clean up code stylistically

* Replace character literal 0 with '\0'.
* Replace pointer literal 0 with NULL.
* Remove redundant parentheses.
* Parentheses in multi-line conditions are not redundant at the
beginning of a line.
* Replace a few !ptr with ptr == NULL.
* Replace a few ptr with ptr != NULL.
* Replace (expr & mask) == 0 with !(expr & mask).
* Remove redundant braces for blocks in cases where the generated code
stays the same. (Assertions further down in the code would get
different line numbers.)
* Rename parameters in CondParser_String to reflect the data flow.
* Replace #ifdef notdef with #if 0.

The generated code stays exactly the same, at least with GCC 5.5.0 on
NetBSD 8.0 amd64 using the default configuration.
 1.195  06-Nov-2020  rillig make(1): rename Arch_MemMTime to Arch_MemberMTime

The abbreviation Mem was ambiguous, it could have meant memory as well.
 1.194  05-Nov-2020  rillig make(1): remove redundant parentheses from sizeof operator

The parentheses are only needed if the argument is a type, not an
expression.
 1.193  31-Oct-2020  rillig make(1): remove dead code for filename expansion using curly braces

Any string containing curly braces is already handled in the very first
if statement.
 1.192  30-Oct-2020  rillig make(1): fix indentation in source code
 1.191  27-Oct-2020  rillig make(1): rewrap and reword the large comment for directory caching

Hashing the directories is not a problem at all, it's the caching that
makes things complicated.
 1.190  26-Oct-2020  rillig make(1): remove "Results: none" from the documentation of void functions
 1.189  25-Oct-2020  rillig make(1): add GNode_Path to access the path of a GNode
 1.188  25-Oct-2020  rillig make(1): rename hash functions to identify the type name

This makes it easier to spot mismatches between the function name and
its first parameter, although the compiler should already catch most of
them. Except for void pointers.
 1.187  25-Oct-2020  rillig make(1): replace Dir_CopyDir with Dir_CopyDirSearchPath

Callback functions for the Lst functions do not belong in the public API
of a module.
 1.186  25-Oct-2020  rillig make(1): clean up Dir_HasWildcards
 1.185  25-Oct-2020  rillig make(1): clean up DirLookupSubdir and DirLookupAbs

As with cp2, the variable names p1 and p2 are not expressive.
 1.184  25-Oct-2020  rillig make(1): omit trailing space in debug output for expanding file patterns
 1.183  25-Oct-2020  rillig make(1): clean up documentation of DirExpandInt, rename it

The code in that function is so trivial that there is no point in
writing this much redundant documentation.
 1.182  25-Oct-2020  rillig make(1): document pattern matching edge case in DirMatchFiles
 1.181  25-Oct-2020  rillig make(1): refactor DirMatchFiles

Split the conditions since the comment only applies to one of them.
Use local variables for common subexpressions.
 1.180  25-Oct-2020  rillig make(1): clean up documentation of DirExpandCurly and Dir_Expand
 1.179  25-Oct-2020  rillig make(1): document that the SearchPath of Dir_FindFile may be NULL
 1.178  25-Oct-2020  rillig make(1): remove unused parameter from DirFindDot
 1.177  25-Oct-2020  rillig make(1): rename parameter of DirLookup

The DirLookup functions work on "name", which may be a complete path,
and on "base" or "cp", which is the basename of the file. Don't use
"name" for the basename, since that would be confusing.
 1.176  25-Oct-2020  rillig make(1): remove unused parameters from DirLookup
 1.175  25-Oct-2020  rillig make(1): only define OpenDirs_Done if necessary
 1.174  24-Oct-2020  rillig make(1): remove UNCONST from Dir_Expand

That code is called so seldom that one more memory allocation doesn't
hurt. It needs a wildcard character in a dependency declaration, which
is rare in practice; see dep-wildcards.mk for an example.
 1.173  24-Oct-2020  rillig make(1): inline DirFindName
 1.172  22-Oct-2020  rillig make(1): remove redundant type casts

This mainly affects the void pointers in callback functions for lists.
These had been necessary once when the parameter type was still
ClientData instead of void pointer.
 1.171  19-Oct-2020  rillig make(1): inline simple Lst getters

The function call variant takes more screen space than the direct field
access. Having an abstract API is usually a good idea, in this case of
simple read-only member access it makes the code more difficult to read.

LstNode_Set has been kept as a function since it is not a read-only
accessor function.
 1.170  18-Oct-2020  rillig make(1): add tags to enum types

This allows IDEs to offer better type information than "anonymous enum".
 1.169  18-Oct-2020  rillig make(1): replace Lst_Open with simple iteration in Dir_SetPATH
 1.168  18-Oct-2020  rillig make(1): replace Lst_Open with simple iteration in Dir_FindFile
 1.167  18-Oct-2020  rillig make(1): rename Lst_Init to Lst_New

For the other types such as HashTable and Buffer, the Init function does
not allocate the memory for the structure itself, it only fills it.
 1.166  18-Oct-2020  rillig make(1): rename HashEntry.name to key
 1.165  18-Oct-2020  rillig make(1): remove underscore from Hash_Table and Hash_Entry

For consistency with the other type names, such as GNodeListNode.
 1.164  18-Oct-2020  rillig make(1): make API for iterating over hash tables simpler
 1.163  17-Oct-2020  rillig make(1): normalize initialization and cleanup of the modules
 1.162  17-Oct-2020  rillig make(1): fix indentation
 1.161  05-Oct-2020  rillig make(1): remove pathname limit for Dir_FindHereOrAbove

While trying to compile the code with GCC's -Wformat-truncation, the
snprintf calls felt quite complicated. The function Dir_FindHereOrAbove
is not in a bottleneck execution path, therefore it doesn't hurt to
dynamically allocate the memory instead of using size-limited stack
memory.
 1.160  05-Oct-2020  rillig make(1): make dir.c, for.c and hash.c ready for WARNS=6

Some types have changed from int to unsigned int, size_t or time_t.

The variable i in hash.c has been kept as int since it counts down to
-1, which generates efficient machine code, at least on x86_64.
 1.159  05-Oct-2020  rillig make(1): fix double-free bug in -DCLEANUP mode (since 2020-10-02)

The bug had been introduced with dir.c 1.155 on 2020-10-02 22:20:25. In
that commit, openDirectories was replaced with a combination of a list
with a hash table, for more efficient lookup by name.

Upon cleanup, OpenDirs_Done is called, which in turn called
Dir_ClearPath. Dir_ClearPath takes full ownership of the given list and
empties it. This was no problem before since afterwards the list was
empty and calling Lst_Free just frees the remaining list pointer.

With OpenDirs, this list was combined with a hash table, and the hash
table contains the list nodes, assuming that the OpenDirs functions have
full ownership of both the list and the hash table. This assumption was
generally correct, except for the one moment during cleanup where full
ownership of the list was passed to Dir_ClearPath, while the hash table
still contained pointers to the (now freed) list nodes. This by itself
was not a problem since the hash table would be freed afterwards. But
as part of Dir_ClearPath, OpenDirs_Remove was called, which looked up
the freed directory by name and now found the freed list node, trying to
free it again. Boom.

Fixed by replacing the call to Dir_ClearPath with code that only frees
the directories, without giving up control over the list.
 1.158  05-Oct-2020  rillig make(1): revert previous commit

It had accidentally reverted all the work from the past few days.
 1.157  05-Oct-2020  rillig make(1): fix double-free bug in -DCLEANUP mode (since 2020-10-02)

The bug had been introduced with dir.c 1.155 on 2020-10-02 22:20:25. In
that commit, openDirectories was replaced with a combination of a list
with a hash table, for more efficient lookup by name.

Upon cleanup, OpenDirs_Done is called, which in turn called
Dir_ClearPath. Dir_ClearPath takes full ownership of the given list and
empties it. This was no problem before since afterwards the list was
empty and calling Lst_Free just frees the remaining list pointer.

With OpenDirs, this list was combined with a hash table, and the hash
table contains the list nodes, assuming that the OpenDirs functions have
full ownership of both the list and the hash table. This assumption was
generally correct, except for the one moment during cleanup where full
ownership of the list was passed to Dir_ClearPath, while the hash table
still contained pointers to the (now freed) list nodes. This by itself
was not a problem since the hash table would be freed afterwards. But
as part of Dir_ClearPath, OpenDirs_Remove was called, which looked up
the freed directory by name and now found the freed list node, trying to
free it again. Boom.

Fixed by replacing the call to Dir_ClearPath with code that only frees
the directories, without giving up control over the list.
 1.156  03-Oct-2020  rillig make(1): clean up #include sections
 1.155  02-Oct-2020  rillig make(1): use hash table for looking up open directories by name

As long as there are less than 20 open directories, it's perfectly fine
to use a doubly-linked list for name lookup. A singly linked list or
even an array list would have been better, but anyway.

When the number of directories rises above 1000, which happens with
dirdeps.mk, linear list lookup becomes too expensive, especially since
each list entry is compared using a strcmp call, in a callback function
that is not inlined.

Using a hash table is much more efficient than linear lookup. While
here, abstract all operations regarding the openDirectories list into a
new data type that provides a simple and straight-forward API. This
strongly typed API is especially important since the current
implementation of the list and hash table is weakly typed, using void *
for the actual data, and StringList and CachedDirList refer to the
exactly same type, they just have different names to help the human
readers but don't provide any type safety.
 1.154  01-Oct-2020  rillig make(1): remove redundant function prototypes
 1.153  28-Sep-2020  rillig make(1): replace += 1 with ++ and -= 1 with --

Just for visual consistency. The generated code stays exactly the same.
 1.152  28-Sep-2020  rillig make(1): make debug logging simpler

This avoids referring to the debug_file variable in many places where
this implementation detail is not necessary.
 1.151  28-Sep-2020  rillig make(1): make debugging code shorter
 1.150  27-Sep-2020  rillig make(1): improve documentation of CachedDir and Dir_AddDir
 1.149  27-Sep-2020  rillig make(1): normalize whitespace in source code

There is no more space tab. Either only tabs or only spaces or tabs
followed by spaces, but not spaces followed by tabs.
 1.148  26-Sep-2020  rillig make(1): inline and remove LstNode_Prev and LstNode_Next

These functions made the code larger than necessary. The prev and next
fields are published intentionally since navigating in a doubly-linked
list is simple to do and there is no need to wrap this in a layer of
function calls, not even syntactically. (On the execution level, the
function calls had been inlined anyway.)
 1.147  25-Sep-2020  rillig make(1): replace a few calls to Lst_Open with simple loops

This avoids relying on the internal iterator of the list, which is
supposed to be removed in the near future.
 1.146  24-Sep-2020  rillig make(1): inline Lst_ForEach for debugging output in search paths
 1.145  24-Sep-2020  rillig make(1): rename Lst_ForEach to Lst_ForEachUntil

Since the callback function returns a terminating condition, this is not
really a foreach loop.

Many of the calls to Lst_ForEachUntil don't make use of the terminating
condition, and several don't modify the list structurally, which means
they don't need this complicated implementation.

In a follow-up commit, Lst_ForEach will be added back with a much
simpler implementation that iterates over the list naively, without a
terminating condition and without taking the iteration state from
Lst_Open/Lst_Next/Lst_Close into account. The migration to this simpler
implementation will be done step by step since each callback function
needs to be examined closely.
 1.144  22-Sep-2020  rillig make(1): use fine-grained type names for lists and their nodes

This is only intended to help the human reader. There is no additional
type safety yet.
 1.143  22-Sep-2020  rillig make(1): rename type Path to CachedDir

The word "path" is commonly used either as an abbreviation for pathname
(a string consisting of several directory or file names) or as an
abbreviation for search path (a list of directory names used for
searching files), but not for a single directory.
 1.142  13-Sep-2020  rillig make(1): clean up RCSID blocks

These blocks mostly consisted of redundant structure, following the same
#ifndef pattern over and over, with only minimal variation.

It's easier to maintain if the common structure is only written once and
encapsulated in a macro.

To avoid "defined but unused" warnings from GCC in the case where
MAKE_NATIVE is not defined, I had to add volatile. Adding
MAKE_ATTR_UNUSED alone would not preserve the rcsid variable in the
resulting binary.
 1.141  12-Sep-2020  rillig make(1): fix assertion failure in Dir_Destroy in -DCLEANUP mode

When the openDirectories path list is cleaned up, each path from it is
first dequeued and then freed via Dir_Destroy. At this point, the path
is no longer in openDirectories, which triggered an assertion in
Lst_Remove, called by Dir_Destroy.
 1.140  12-Sep-2020  rillig make(1): fix indentation in dir.c
 1.139  12-Sep-2020  rillig make(1): rename local variable in Dir_FindFile

The name "cp" is not appropriate for a variable containing the basename
of a path.
 1.138  11-Sep-2020  rillig make(1): replace *a->b with a->b[0]

This allows the code to be read strictly from left to right. In most
places this style was already used.
 1.137  07-Sep-2020  rillig make(1): document that nested braces work as expected now
 1.136  05-Sep-2020  rillig make(1): remove initial size argument from Hash_InitTable

In all but one case this argument was set to auto-detect anyway. The
one case where it was set was not worth keeping this complicated API.
 1.135  02-Sep-2020  rillig make(1): fix cached_stat for files with st_mtime 0
 1.134  02-Sep-2020  rillig make(1): fix aliasing problem in cached_stat from the previous commit

When the struct stat was used for both calling the actual stat and for
returning the result, no copying was needed. This also had the side
effect that for the first call of cached_stat, the returned struct stat
included all the fields properly filled in, and on later calls, these
fields were all zeroed out.

These two variables are separate now, thus the fields need to be copied
explicitly. There are no existing unit tests for this, but ./build.sh
failed reliably.
 1.133  02-Sep-2020  rillig make(1): reduce number of stat fields returned by cached_stat

Only st_mtime and st_mode are actually filled, the remaining fields had
been set to zero. To prevent these from ever being accessed, a custom
struct make_stat replaces the previously used struct stat.

The fields in struct make_stat are intentionally named different from
the fields in struct stat because NetBSD and some other operating
systems define st_mtime as a macro, and that would not work in a field
declaration.
 1.132  02-Sep-2020  rillig make(1): use proper types in API of cached_stat and cached_lstat
 1.131  02-Sep-2020  rillig make(1): use Hash API from dir.c

When the Hash struct fields are renamed the next time, this should not
influence any code outside hash.h and hash.c.
 1.130  01-Sep-2020  rillig make(1): rename Hash_Table fields

Back in the 1980s it made sense to have the type information encoded in
the variable names. At the time when make was imported into the NetBSD
tree (1993-03-21), the functions did indeed not have prototypes, they
only had return types. The void type was already invented at that time.
Since the compiler could not verify the types of function parameters, it
made perfect sense to have each variable tell whether it was a pointer
or not.

Since ISO C90 this is no longer necessary since the compiler checks
this. The variable names can now focus on the application level and
their high-level meaning, expressing the relationship to other
variables instead of encoding redundant type information.
 1.129  01-Sep-2020  rillig make(1): improve variable names and data types in Dir_FindHereOrAbove
 1.128  01-Sep-2020  rillig make(1): make data types in Dir_HasWildcards more precise
 1.127  30-Aug-2020  rillig make(1): rename GNode.iParents to implicitParents

The i alone was too ambiguous. It could have meant ignore, implicit,
interactive, and probably many more.
 1.126  30-Aug-2020  rillig make(1): rename Lst_Datum to LstNode_Datum
 1.125  30-Aug-2020  rillig make(1): rename Lst_Memeber to Lst_FindDatum

The new name nicely aligns with Lst_Find and Lst_FindFrom.
 1.124  29-Aug-2020  rillig make(1): clean up comments in dir.c
 1.123  29-Aug-2020  rillig make(1): rename LstNode functions to match their type
 1.122  29-Aug-2020  rillig make(1): rename Lst_FindB back to Lst_Find

The migration from "comparison function" to "match function" is done,
the "B" in the names is no longer needed.
 1.121  29-Aug-2020  rillig make(1): start replacing Lst_Find with Lst_FindB

Lst_Find is called with a "comparison" function that returns the integer
0 if the desired node is found. This leads to confusion since there are
so many different return value conventions for int, such as 0/1 for
mimicking false/true, -1/0 as in close(2), and the sign as in strcmp(3).
This API is much easier to understand if the "comparison" function is
not called a comparison function (since that is too close to strcmp),
but a "match" function that just returns a boolean.

In Lst_FindFromB, the node argument may be null. This deviates from the
other Lst functions, which require Lst and LstNode to generally be
non-null. In this case it is useful though to make the calling code
simpler.

In arch.c, this makes a lot of the previous documentation redundant.

In cond.c, the documentation is reduced a little bit since it had
already been cleaned up before. It also removes the strange negation
from CondFindStrMatch.

In dir.c, the documentation collapses as well.

In main.c, separating the ReadMakefile function from the callbacks for
Lst_FindB allows the former to get back its natural function signature,
with proper types and no unused parameters.

To catch any accidental mistakes during the migration from Lst_Find to
Lst_FindB, the code can be compiled with -DUSE_DOUBLE_BOOLEAN, which
will complain about incompatible function pointer types.
 1.120  28-Aug-2020  rillig make(1): clean up Dir_AddDir

Extract the null check for path to the top level. This has the
side-effect of only incrementing dotLast.refCount if that entry is
actually used.

Reduce the indentation of the code by returning early from the simple
branches.
 1.119  28-Aug-2020  rillig make(1): remove trailing 'S' from names of Lst functions

The migration from null-passing Lst functions to argument-checking Lst
functions is completed.

There were 2 surprises: The targets list may be NULL, and in Dir_AddDir,
the path may be NULL. The latter case is especially surprising since
that function turns into an almost-nop in that case. This is another
case where probably 2 independent functions have been squeezed into a
single function. This may be improved in a follow-up commit.

All other lists were fine. They were always defined and thus didn't
need much work.
 1.118  28-Aug-2020  rillig make(1): migrate Lst_Find to Lst_FindS
 1.117  28-Aug-2020  rillig make(1): remove unused reference to Lst_Last
 1.116  28-Aug-2020  rillig make(1): migrate Lst_First to Lst_FirstS
 1.115  27-Aug-2020  rillig make(1): migrate Lst_IsEmpty to Lst_IsEmptyS
 1.114  27-Aug-2020  rillig make(1): migrate Lst_Succ to Lst_SuccS
 1.113  27-Aug-2020  rillig make(1): migrate Lst_ForEach to Lst_ForEachS

Most lists are always valid. Only the "targets" variable may be null in
some cases, probably.
 1.112  27-Aug-2020  rillig make(1): migrate remaining code from Lst_Open to Lst_OpenS
 1.111  26-Aug-2020  rillig make(1): add stricter variants for remaining Lst functions

In most cases the Lst functions are only called when the arguments are
indeed valid. It's not guaranteed though, therefore each function call
needs to be analyzed and converted individually.

While here, remove a few statements that were only useful when the Lst
functions handled circular lists.
 1.110  23-Aug-2020  rillig make(1): reverse order of the Lst_Find parameters

The other callbacks all have (function, param), only the Lst_Find had
(param, function), which was inconsistent.
 1.109  22-Aug-2020  rillig make(1): migrate Lst_AtFront to Lst_PrependS

This makes Lst_AtFront unused, as well as LstInsertBefore.
 1.108  22-Aug-2020  rillig make(1): replace Lst_Duplicate with Lst_CopyS

Lst_Duplicate would have passed through any null pointer, which was not
needed for make. It was the last function that used Lst_AtEnd, which in
turn was the last function that used LstInsertAfter. As a result, these
two functions have been removed.
 1.107  22-Aug-2020  rillig make(1): use Lst_OpenS in Dir_SetPATH

Since dirSearchPath is initialized in Dir_Init, opening the list can
never fail.
 1.106  22-Aug-2020  rillig make(1): fix indentation
 1.105  22-Aug-2020  rillig make(1): extract percentage calculation out of Dir_PrintDirectories
 1.104  22-Aug-2020  rillig make(1): require argument of Lst_Member to be non-null

Since the lists don't contain null pointers, it doesn't make sense to
search for a null pointer. All calls but one already had obviously
non-null arguments. The one remaining call using targ->suff has been
guarded for now.

The code for Lst_Member became much simpler than before. Partly because
the old code had an extra condition for circular lists, which are not
used by make.
 1.103  22-Aug-2020  rillig make(1): replace Lst_Datum with non-null guaranteeing Lst_DatumS
 1.102  22-Aug-2020  rillig make(1): convert Lst_Enqueue and Lst_Dequeue to nonnull variants

Except for once instance in parse.c, the usage pattern for Lst_Dequeue
was to first test whether the list is empty. This pattern allowed the
implementation of Lst_Dequeue to become simpler since the null check is
not needed anymore.

The calls to Lst_Enqueue never pass an invalid list or a null pointer,
therefore making them strict was trivial.
 1.101  22-Aug-2020  rillig make(1): remove wrong comment for Dir_Init

Neither the list nor the hash table modules have anything to do with
opening directories.
 1.100  22-Aug-2020  rillig make(1): replace "(void)Lst_AtEnd" with stricter "Lst_AppendS"

This change ensures that there is actually something added to the list.
Lst_AtEnd had silently skipped the addition if the list was invalid
(null pointer), which was not intended in these cases. The "(void)" is
assumed to mean "I know that this cannot fail", while it could also mean
"I don't care whether something actually happened".

Running "./build.sh -j6 tools" still succeeds after this change,
therefore chances are very low that this change breaks anything. If
there is any change, it's an obvious assertion failure. There is no
silent change in behavior though.
 1.99  22-Aug-2020  rillig make(1): add Lst_Append to add an item at the end of the list

The previous variant of using a special case of Lst_InsertAfter was
unnecessarily complicated. Linked lists are a very basic data
structure, and there is no need to overcomplicate things by introducing
unnecessary conditions and branches.
 1.98  22-Aug-2020  rillig make(1): remove unused return value for DirMatchFiles
 1.97  22-Aug-2020  rillig make(1): split Dir_Init into two functions

There's just no point in having a function consisting of a big
if-then-else.
 1.96  21-Aug-2020  rillig make(1): use stricter list API for sequential access

In several places, it just doesn't make sense to have a null pointer
when a list is expected.

In the existing unit tests, the list passed to Lst_Open is always valid,
but that's not a guarantee for real-world usage. Therefore, Lst_Open
has been left for now, and Lst_OpenS is only the preferred alternative
to it.
 1.95  21-Aug-2020  rillig make(1): assert correct usage of the Lst_Open API

All calls to Lst_Next are properly protected by Lst_Open, so there is no
possible assertion failure here.
 1.94  21-Aug-2020  rillig make(1): make list library code stricter

Up to now, the list library didn't distinguish between programming
mistakes (violations of invariants, illegal parameter values) and
actually interesting situations like "element not found in list".

The current code contains many branches for conditions that are neither
exercised by the unit tests nor by real-world usage. There is no point
in keeping this unnecessary code.

The list functions will be migrated from their lenient variants to the
stricter variants in small parts, each function getting the S suffix
when it is made strict, to avoid any confusion about how strict a
particular function is. When all functions have been migrated, they
will be renamed back to their original names.

While here, the comments of the functions are cleaned up since they
mention irrelevant implementation details in the API comments, as well
as "side effects" that are really main effects.
 1.93  21-Aug-2020  rillig make(1): remove unused code for circular lists

The list library had probably been imported from a general-purpose
library that also supported circular lists. These are not used by make
though.

After replacing Lst_Init(FALSE) with Lst_Init(), only a single call to
Lst_Init remained with a non-constant argument, and that was in
Lst_Concat, which was to be expected.
 1.92  13-Aug-2020  rillig make(1): clean up debug logging in dir.c

In C90, the variadic macro argument __VA_ARGS__ is not known, therefore
fall back to the forms listing the number of actual printf arguments.
 1.91  13-Aug-2020  rillig make(1): avoid negated conditions in DirExpandCurly
 1.90  13-Aug-2020  rillig make(1): clean up DirExpandCurly

Now that nested curly braces work as expected and are covered by unit
tests, the debug log is no longer necessary.
 1.89  13-Aug-2020  rillig make(1): use enum instead of #define for cached_stats
 1.88  12-Aug-2020  rillig make(1): make Dir_MakeFlags simpler

This avoids unnecessary string allocations, especially for long lists.

There is no unit test to cover this code. Since this is an obscure,
undocumented part of make, I wasn't able to come up with a unit test.
Therefore I resorted to write a separate testing program to verify the
expected results.

$ cat <<'EOF' >dir_test.c
#include <stdio.h>

#include "make.h"
#include "dir.h"

int main(int argc, char **argv)
{
int i;
Lst lst;
char *flags;

lst = Lst_Init(FALSE);
for (i = 1; i < argc; i++)
Dir_AddDir(lst, argv[i]);
flags = Dir_MakeFlags("-I", lst);

printf("%s\n", flags);
free(flags);
return 0;
}
EOF

# Needs some trivial patches to Makefile and main.c
$ make PROG=dir_test EXTRA_SRCS=dir_test.c EXTRA_CPPFLAGS=-DNO_MAIN \
COPTS.main.c=-Wno-unused-function NOMAN=1

$ ./dir_test a b c

$ mkdir a b c
$ ./dir_test a b c
-Ia -Ib -Ic
$ rmdir a b c
 1.87  10-Aug-2020  rillig make(1): replace str_concat with str_concat2 and str_concat3

The new functions have a simpler interface, and str_concat3 is even more
general-purpose, since the middle string is no longer required to be
exactly of length 1.
 1.86  10-Aug-2020  rillig make(1): fix parameter name of str_concat

The previous documentation mentioned Str_Concat, but str_concat has been
written in lowercase for years. The "flags" are not flags since they
cannot be combined, not even when they are written in hex.
 1.85  09-Aug-2020  rillig make(1): format the source code consistently, at least per file

Some files use 4 spaces per indentation level, others use 8. At least
for the few files from this commit, they use a consistent style
throughout each file now.

In Cond_Eval, the #define has changed into an enum since the identifiers
need not be visible to the C preprocessor.
 1.84  03-Aug-2020  rillig make(1): no declaration-after-statement anymore

NetBSD make is intended to be maximally portable, therefore it uses only
C89. This was not declared in the Makefile before.

There are still a few places in parse.c and metachar.c that use
end-of-line comments. These will be fixed in a follow-up commit.
 1.83  01-Aug-2020  rillig make(1): use consistent indentation in source code

Tabs for multiples of 8, then spaces.

The usage string has been kept as-is since the spaces there are
indentional and do influence the output.
 1.82  31-Jul-2020  rillig make(1): document another instance of undefined behavior

The UNCONST macro is really terrible.

This segmentation fault can be forced by setting _PATH_DEFSYSMK in
pathnames.h to "./sys*.mk" or any other string that has a slash and a
wildcard to the right of the slash.
 1.81  31-Jul-2020  rillig make(1): fix regression and add test for empty brace expansion

This had been broken in r1.80 of dir.c, a few minutes ago.
 1.80  31-Jul-2020  rillig make(1): refactor DirExpandCurly

Separating the low-level parts into small functions reduces the need for
summarizing comments between the code lines.

Using a consistent naming scheme for the variables and expressive names
makes the code easier to understand. The number of variables has
increased from 7 to 11, their clearer names compensate for that, plus
the fact that they come in triples (x, x_end, x_len). Placing the
variables into appropriate registers and eliminating memory access is
left as an exercise to the compiler.
 1.79  31-Jul-2020  rillig make(1): fix parsing of nested braces in dependency lines

Before, make could not parse {{thi,fou}r,fif}teen properly. It did
correctly split up the outer brace into "" + "{thi,fou}r,fif" + "teen",
but then, when expanding the inner braces, it interpreted the first
comma already as a separator, even though this comma was enclosed in
another set of braces.

This resulted in the wrong expansion "{thiteen", which produced the
error message. The next word "fouteen" was produced since the parser
stopped at the next closing brace. After this, parsing continued after
the closing brace, producing "rteen". Finally, the last expansion was
the correct "fifteen".
 1.78  31-Jul-2020  rillig make(1): trying to understand how DirExpandCurly works
 1.77  31-Jul-2020  rillig make(1): prefer memcpy over strncpy in DirExpandCurly

strncpy has unnecessary overhead when the source memory is already
guaranteed to contain no '\0'.
 1.76  03-Jul-2020  rillig make(1): remove trailing whitespace
 1.75  03-Jul-2020  rillig make(1): remove redundant parentheses around return values
 1.74  05-Jun-2020  sjg cached_stats: do not conflate stat and lstat

While make uses lstat quite rarely, it does so for reason.
Avoid confusing the results.

Reviewed by: christos
 1.73  12-Jul-2018  christos Fix previous: cached_stats() returning < 0 means that the file is not found,
not that it was found in the cache, and centralize reporting.
 1.72  12-Jul-2018  reinoud Remove duplicate code in make(1)'s dir.c.

When the cached_stats() code was added, some old logic stayed around that
implements the cached_stats() too.
 1.71  16-Apr-2017  riastradh branches: 1.71.10; 1.71.12;
Plug memory leaks in Dir_FindFile.

CID 978364
 1.70  16-Apr-2017  riastradh Use, don't kludge, MAKE_ATTR_UNUSED.

CID 1300234
CID 1300237
CID 1300238
CID 1300245
CID 1300255
CID 1300267
CID 1300284
 1.69  31-Jan-2017  sjg Partially initialize Dir before MainParseArgs can be called.

The rest can be done once curdir is finalized.
 1.68  07-Jun-2016  sjg branches: 1.68.2; 1.68.4;
Extend the mtimes cache used by dir.c so it can be used by others.

We store both st_mtime and st_mode, since some callers care about the
later.

Reviewed by: christos
 1.67  05-Mar-2013  christos Add a .STALE special target that gets invoked when dependency files contain
stail entries.
 1.66  05-Mar-2013  christos Keep track of the location where a dependency is defined, so we can report
about it.
 1.65  12-Jun-2012  joerg branches: 1.65.2;
Replace __dead, __unused and the various printf format attributes
with versions prefixed by MAKE_ATTR_* to avoid modifying the
implementation namespace. Make sure they are available in all places
using nonints.h to fix bootstrap on Linux.
 1.64  07-Apr-2012  christos Remove recheck hackery that caused extra stats, and explicitly ask for
recheck when needed. Before it used to be the case that we could only
use the cached entry once. Once the cached entry was used, we removed
it from the cache. Now it is kept forever.
 1.63  05-Mar-2011  sjg branches: 1.63.4;
Treat some/dir/ the same as some/dir/.
 1.62  27-Nov-2010  sjg When a source file moves, make will ignore the stale dependency,
but if the file in question is one that needs to be compiled (.c or .cc),
it still hands the bogus name to the compiler.

If Dir_MTime() cannot find such a file (gn->iParents is not empty),
see if the basename can be found via .PATH, and if so set gn->path to
the found file. This prevents the stale path being given to the
compiler.

In meta_oodate(), if a referenced file no longer exists, consider the
target out-of-date.

Also, if meta_oodate() decides a target is out-of-date, and it
it uses .OODATE in its commands, we need .OODATE recomputed.
Undo our call to Make_DoAllVar() so that the call from Make_OODate()
will do the right thing.
 1.61  24-Jan-2009  dsl Don't cast 'time_t' to 'void *' and back it will lose precision.
 1.60  23-Jan-2009  dsl Sprinkle some const.
In particular for Lst_Find() and Lst_FindFrom().
Remove some unneeded casts and some now-undeeded UNCONST().
 1.59  23-Jan-2009  dsl Change 'ClientData' to 'void *' so that relevant parameters can
be made 'const void *'.
 1.58  28-Dec-2008  christos if mtime == 0, make it 1 because the code expects 0 time to mean that the
file does not exist
 1.57  13-Dec-2008  dsl Use NULL instead of -1 cast to the relavant type (usually via NIL).
This was a suggestion from christos - so blame him if there is a deep
reason for using -1 :-)
 1.56  06-Oct-2008  joerg Don't use emalloc and friends directly, but call them consistently
bmake_malloc and friends. Implement them via macros for the native case
and provide fallback implementations otherwise. Avoid polluting the
namespace by not defining enomem globally. Don't bother to provide
strdup and strndup, they were only used for the estrdup and estrndup
comapt code.

This addresses the presence of emalloc in system libraries on A/UX and
resulted strange issues as reported by Timothy E. Larson.
 1.55  15-Feb-2008  christos back all changes out until I fix it properly.
 1.54  14-Feb-2008  christos - use pid_t/size_t as appropriate instead of int.
- use %ld to print pids.
- fix a bit of lint.
- WARNS=4
 1.53  01-Jan-2007  dsl Remove 'else' after 'return', and similar whitspace changes.
Extra debug on where files are actually found.
 1.52  16-Dec-2006  dsl Fix a comment that has been wrong since rev 1.1, minor layout fix.
 1.51  27-Oct-2006  dsl branches: 1.51.2; 1.51.4;
Since 'ClientData' is 'void *', nuke almost all the (ClientData) casts.
 1.50  15-Oct-2006  dsl Output all debug trace output through 'debug_file' defaulting to 'stdout'.
(Almost all the debug output went there, but some went to stderr.)
Split the parsing of -d (debug flags) out into its own routine.
Allow the output filename to be changed by specifying -dF<file> to create
a log file, or -dF+<file> to append to it. <file> may be stdout or stderr.
Also change so that -d-<flags> acts on <flags> locally but doesn't copy
them to MAKEFLAGS so they aren't inherited by child makes.
I'm not 100% happy with the command line syntax for the above, so they are
currently undocumented.
 1.49  22-Apr-2006  christos Coverity CID 528: Avoid NULL deref.
 1.48  08-Aug-2005  christos From Max Okumoto:
- Remove casts to NULL.
- Remove space between cast and object.
 1.47  05-Aug-2005  christos More KNF cleanups from Max Okumoto
 1.46  25-Jul-2005  christos Whitespace KNF cleanup from Max Okumoto
 1.45  16-Feb-2005  christos PR/29203, PR/29204: Max Okumoto: KNF changes to make [no functional changes]
 1.44  29-Dec-2004  christos Dir_MTime did not search for a file using the correct parh; i.e. it was
ignoring suffix-specific path search. So if a node was marked .MADE,
then suffix rules would not be applied to it, and we would look for
the file only in the default path, not the suffix-specific path.

XXX: Now that we looked for the suffix, we can save it in the GNode,
but we don't do this yet.
 1.43  01-Jul-2004  jmc Change to use __unused instead and provide a compat definition in make.h if
not already defined from cdefs.h
 1.42  01-Jul-2004  jmc Add some checks for gcc around a few function declarations and note the
unused variables. Also fix a few other warnings that PR#22118 shows when
trying to compile bmake on non-NetBSD hosts
 1.41  07-May-2004  ross Simplify build, no functional changes.

Instead of adding MAKE_BOOTSTRAP for hosted environments, i.e., when
you want things simple, instead add MAKE_NATIVE to get those hugely
important features like __RCSID().

It's now possible to build make on some hosts with: cc *.c */*.c
 1.40  03-Feb-2004  chuck branches: 1.40.2;
add parent directory search for make as discussed on tech-toolchain.
- new dir.c function: Dir_FindHereOrAbove:
Search for a path in the current directory and then all the directories
above it in turn until the path is found or we reach the root ("/").
- add hooks to use it in main.c for -m and syspath (compiled in
_PATH_DEFSYSPATH and $MAKESYSPATH).
- updated man page
 1.39  11-Jan-2004  dsl Change DEBUG(DIR) traces to indent subsequent filenames instead of
generating very long lines (basically changes "..." into " ...\n ").
 1.38  09-Sep-2003  drochner Remove some code which makes file lookup rely on the fact that
the first two directory entries are "." and "..".
This behaviour is not required by applicable standards, and
actually not provided by "coda".
Now we get the "." and ".." into the per-directiry hash tables,
but this should not hurt.
 1.37  07-Aug-2003  agc Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22365, verified by myself.
 1.36  14-Jul-2003  christos Pass WARNS=3
 1.35  26-Nov-2002  sjg Whenever we update .PATH, set the variable ${.PATH} to reflect the
search list that will be used. Thus 'dot' and 'cur' will appear in
${.PATH} either at the start or end depending on .DOTLAST even though
they are not strictly in dirSearchPath.

When .CURDIR is assigned to - re-set the 'cur' Path.

Finally, when checking subdirs, look in 'dot' and 'cur' (first or last
depending on .DOTLAST) just as we do in other situations.
 1.34  15-Jun-2002  wiz Remove !__STDC__ stuff, de-__P(), ANSIfy, and de-register.
 1.33  03-Feb-2002  pk Resurrect revision 1.23: no need to stat .PHONY targets.
 1.32  31-Jan-2002  pk Fix the bug addressed in revision 1.27 properly. Analysis of the problem
(see also PR#15179):

When looking up names which directory components (i.e. having slashes,
except when of the form `./name'), FindFile()/DirLookup() first looks
the final filename component in the cache for each directory on the search
path and then proceeds to match the prefixed directory components by
comparing them to the trailing directory components of the the search
path being probed.

This is not correct. When looking for `bar/target' in a path `.../src/foo',
you want it to come up with `.../src/foo/bar/target' (if it exists). There's
no point in comparing the the `bar' prefix on the target to the `foo' suffix
on the search path. Indeed, this will cause a false match if those prefix
and suffix components are actually equal and search path itself also has a
file called `target'. For example, looking for `foo/target' in `.../src/foo'
will spuriously match `.../src/foo/target', not `.../src/foo/foo/target'.

This last bug prompted the change in dir.c, rev 1.27, which happens
to partially workaround it by avoiding the above matching code in the
case of the `curdir' search path entry (at the cost of incurring an
exorbitant amount of cache misses). The situation is unchanged however,
when processing other entries on the search path (e.g. those other than
`dot' and `cur').

Drop the prefix matching code in DirLookup() entirely and use DirFindDot()
and DirLookup() only for names without proper directory components (i.e.
`target' and `./target). Otherwise, non-absolute names are dealt with by
DirLookupSubdir(), while absolute names can be checked for an exact match
of the directory components prefix against the directories on the current
search path. This allows for the use of the file cache to check the
existence of the file and additionally, provides a shortcut out of
Dir_FindFile() if we have the prefix match but not a cache entry (this
is especially beneficial for searches in .CURDIR when it's not equal
to `dot').
 1.31  27-Jan-2002  reinoud Fix major bug in make(1) ... due to shadowing of the dotLast path used for
the .DOTLAST primitive by a boolean variable with the same name, this whole
mechanism was broken ... it doesn't save much stat calls but it was wrong.

Thanks to Jason Thorpe for the other shadow-variable fixing patches he
made.
 1.30  26-Jan-2002  christos Don't bother to lookup for files in `.' that start with a `/'. Cuts in the
number of NAMI's from 1344 -> 830 in a full build of /usr/src/usr.bin/make.
 1.29  18-Jan-2002  pk In Dir_FindFile() add a missing Lst_Close() and move another one to the
proper place.
 1.28  12-Nov-2001  tv Redo the hashtable for "." if .OBJDIR changes.
 1.27  11-Nov-2001  tv Fix long-standing bug where, if an objdir exists, searching the .PATH for a
file will strip any leading path on the filename when searching ${.CURDIR}.

This bug manifested itself more prominently with xsrc/xfree/xc/lib/GL/GL,
since we now always search ${.CURDIR} as an alternative to ${.OBJDIR}
(PR bin/14499).
 1.26  16-Apr-2000  christos Don't core-dump when the current working directory is not readable
 1.25  25-Nov-1999  mrg back out rev 1.23 (don't stat PHONY targets). it broken the libcrypto
build by causing everything to be remade everytime. this fixes PR#8419.
 1.24  15-Sep-1999  mycroft branches: 1.24.4;
Don't bother iterating through all the data structures to free(3) everything
right before exiting.
(The code is still present, `#ifdef CLEANUP', in case someone needs it...)
 1.23  15-Sep-1999  mycroft Fix another case where we stat(2)ed .PHONY targets.
 1.22  12-Jul-1999  thorpej Correct an inverse initialization of a variable, which caused all .PATHs
to be treated as .DOTLAST.
 1.21  11-Jul-1999  thorpej Add a mechanism for specifying that ${.CURDIR} will be searched last
in the presence of .PATH directives by specifying:

.PATH: .DOTLAST

This will be used to fixup the build system to work with both crypto-us
and crypto-intl sub-trees.

Make(1) changes by Christos Zoulas, after much badgering by me :-)
 1.20  28-Sep-1997  lukem wrap #include <sys/cdefs.h>, __RCSID(...) stuff in #ifndef MAKE_BOOTSTRAP
 1.19  01-Jul-1997  christos Add WARNS=1
RCSID police
 1.18  09-May-1997  christos Change the way curdir is handled [this started at the previous version]...
Instead of adding it in .PATH, handle it specially the same way as "." is
handled.
 1.17  08-May-1997  gwr Add the new .NOPATH feature which can be used to disable .PATH search
for particular targets, i.e. .depend, objects, etc. (from Christos).
 1.16  06-May-1997  mycroft Don't do a VPATH or .PATH search for .PHONY targets. (From Christos.)
 1.15  02-May-1997  christos Make used to add every directory where it found files in the search path.
I.e. if you had a line in your Makefile:
../foo.o: foo.c
`..' would be added in the search path. The addition of such paths has
been now disabled. If a pathname contains a slash, then the directory
where such a file is found is not added to the search path. Of course
this eliminates most (all?) use of this function.
 1.14  29-Mar-1997  christos Don't disable wildcards completely; they are used by other Makefiles.
 1.13  27-Mar-1997  christos - Disable globbing for targets/dependencies when POSIX is defined.
- Fix globbing so that patterns that don't have a matching number of [] or {}
don't get expanded. (before the [ case got expanded to nothing!) This is
disabled.
 1.12  06-Nov-1996  christos - Merge in FreeBSD and Lite2 changes.
- Fix bug where a non-archive target with a .a suffix would always
be considered to be out of date, since it does not have a TOC.
 1.11  13-Aug-1996  christos Add estrdup(), a checked version of strdup and use it.
 1.10  04-Feb-1996  christos branches: 1.10.4;
fix pr/1421 and pr/1997
 1.9  22-Nov-1995  christos Updates for POSIX/SVR4 compiling:

arch.c: Don't require ranlib stuff. Not everybody has it.
dir.c: SunOS-4 != Solaris; change #ifdef sun to #if sun && !__svr4__
job.c, compat.c: Don't use 'union wait', use int and the W*() macros.
main.c: Check for uname() == -1; some unames return > 0...
util.c, job.c: Add signal() with BSD semantics for svr4, don't use bsd
sigmask and friends.
 1.8  14-Jun-1995  christos - $NetBSD$ rcsids
- Fixed so that .[A-Z]* targets that do not match keywords are ignored as
Posix mandates
- Added .PHONY target keyword
 1.7  24-Dec-1994  cgd a few casts, for peace and quiet.
 1.6  06-Jun-1994  jtc Fixes from Christos Zoulas, who used purify, objectcenter and testcenter
to find memory leaks and illegal memory accesses.
 1.5  05-Mar-1994  cgd fixes/improvements from Christos Zoulas <christos@deshaw.com>.
 1.4  13-Jan-1994  jtc Include appropriate header files to bring prototypes into scope.
 1.3  08-Dec-1993  jtc Update to use <dirent.h>.
 1.2  01-Aug-1993  mycroft Add RCS identifiers.
 1.1  21-Mar-1993  cgd branches: 1.1.1;
Initial revision
 1.1.1.2  28-Dec-1996  tls Import 4.4BSD-Lite2 sources onto CSRG branch (already merged at head)
 1.1.1.1  21-Mar-1993  cgd initial import of 386bsd-0.1 sources
 1.10.4.1  26-Jan-1997  rat Update make(1) from trunk, by request from Christos Zoulas. Fixes many bugs.
 1.24.4.1  27-Dec-1999  wrstuden Pull up to last week's -current.
 1.40.2.1  10-May-2004  tron Pull up revision 1.41 (requested by sjg in ticket #282):
Simplify build, no functional changes.
Instead of adding MAKE_BOOTSTRAP for hosted environments, i.e., when
you want things simple, instead add MAKE_NATIVE to get those hugely
important features like __RCSID().
It's now possible to build make on some hosts with: cc *.c */*.c
 1.51.4.1  03-Sep-2007  wrstuden Sync w/ NetBSD-4-RC_1
 1.51.2.1  05-Jun-2007  bouyer Apply patch (requested by tron in ticket #696):
usr.bin/make/compat.c patch
usr.bin/make/cond.c patch
usr.bin/make/dir.c patch
usr.bin/make/for.c patch
usr.bin/make/main.c patch
usr.bin/make/make.1 patch
usr.bin/make/make.c patch
usr.bin/make/make.h patch
usr.bin/make/nonints.h patch
usr.bin/make/parse.c patch
usr.bin/make/str.c patch
usr.bin/make/targ.c patch
usr.bin/make/util.c patch
usr.bin/make/var.c patch

Synchronize make(1) with HEAD branch to increase perfomance and
improve stability.
 1.63.4.3  22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.63.4.2  30-Oct-2012  yamt sync with head
 1.63.4.1  17-Apr-2012  yamt sync with head
 1.65.2.1  23-Jun-2013  tls resync from head
 1.68.4.1  21-Apr-2017  bouyer Sync with HEAD
 1.68.2.2  26-Apr-2017  pgoyette Sync with HEAD
 1.68.2.1  20-Mar-2017  pgoyette Sync with HEAD
 1.71.12.1  10-Jun-2019  christos Sync with HEAD
 1.71.10.1  28-Jul-2018  pgoyette Sync with HEAD

RSS XML Feed