make: clean up

make: clean up

Replace 'unsigned int' with simply 'unsigned'.

In compat.c, skipping whitespace is not needed, as the loop above
already skips it.

In job.c, remove the unused header <sys/file.h>.

Inline the TMPPAT macro, as it is only needed in a single place.

make: group the code for handling the job token pool

make: move struct Job from job.h to job.c

The content of this struct is an implementation detail, and other parts
of make only need to access very few parts of it.

make: clean up job handling

make: only generate code for cleanup functions in CLEANUP mode

branches: 1.79.2;
make: reduce usage of the UNCONST hack

make: clean up comments

No binary change, except for line numbers in assertions.

make: format comments according to /usr/share/misc/style

Assisted by indent(1), with manual corrections due to its many remaining
bugs.

No functional change.

make: use consistent indentation for statements and continuations

No binary change, except for line numbers in assertions in suff.c.

make: remove redundant comments for multiple-inclusion guards

make: mark several functions whose result must be used

Suggested by sjg, to catch more bugs like the memory leak in cond.c
1.303 from 2021-12-13.

No binary change.

make: use C99 bool type instead of defining its own

No functional change.

Avoid strdup in mkTempFile

Require caller to pass a buffer and size if they
want the tempfile not unlinked.

Add Job_TempFile to handle blocking signals around
call to mkTempFile, so that meta_open_filemon can use it
in jobs mode.

make(1): format multi-line comments

make(1): indent job.h with tabs instead of spaces

make(1): move Job.xtraced to ShellWriter

This flag was placed wrong in the Job since it is only necessary as long
as the shell commands are written to the shell file.

Resetting it in JobStart and JobExec was completely misguided since that
is far away from writing the shell commands; this should have been done
in JobPrintCommands instead.

The status of this flag doesn't need to be printed in debugging mode
since it is controlled by a single command line option (-dx) and does
not interact with all the other switches.

make(1): in jobs mode, extract writing of shell commands

Right now, the test sh-flags.mk demonstrates many variants to configure
echoing of the shell commands (-s, .SILENT, '@'), error handling (-i,
.IGNORE, '-') and whether the commands are run (-n, -N, .MAKE,
.RECURSIVE, '+').

Even more variants are possible by configuring the shell to have error
control. None of the built-in shell definitions has error control, so
it is unlikely that anybody uses them, but who knows.

Being able to configure these details at 3 levels is good, but what
makes all this really hard to understand is that some of these switches
interact in non-obvious ways. For example, in jobs mode, a single
command can change job->ignerr (in JobPrintSpecialsEchoCtl), which will
affect all further commands of that job.

The goal of this refactoring is to make the code easier to understand by
making the switches on the job level constant and by moving all
modifications to them to the ShellWriter.

make(1): negate Job.silent to echo

This gets rid of a few explicit negations and an implicit negation in
the variable name.

make(1): unpack struct JobFlags

The job flags are only used individually.

make(1): split JobFlags into separate fields

Having all these flags in a single bitmask makes it harder to see where
exactly they can possibly be used since their state could also be
modified using the unsuspicious job->flags = 0. Using individual names
just leaves the single memset, and that is only used during
initialization.

make(1): use space instead of tab for preprocessor directives

make(1): fix grammar: setup -> set up

make(1): rename JobState to JobStatus

The status is only part of the whole state of a job.

make(1): remove unused JOB_IGNDOTS

It had been used in 1993 at least, don't know when it was finally
removed.

make(1): fix type of Job.suspended

make(1): clean up JobStart

Initialize the fields in declaration order.
Initialize Job.flags in a single assignment.

make(1): group the command line options and arguments

By having a single struct that holds all command line options and
arguments, it is easy to see in the code when such a command line
argument is modified. It also cleans up the namespace since the command
line options don't follow a common naming style. Having them in a
struct also means that there is a single place for putting the
documentation, not two as before.

The struct also suggests to extract the initialization code out of main,
which is still too large, having more than 400 lines of code and
covering far too many topics.

make(1): convert JobState and JobFlags to enum types

Both GCC and Clang complained when JobFindPid had its parameter "status"
as an int. Strangely both compilers complained about a comparison
between unsigned and signed int, even though enums are defined to be
int, not unsigned.

make(1): move struct Shell from job.h to job.c

This type is only needed in job.c.

make(1): prepare job.c, main.c, parse.c, suff.c for WARNS=6

In job.c, GCC 5 complains about the macro FILENO that it has type
unsigned int, which is then passed as the argument of dup2, which
expects a simple int. Maybe this workaround from 1995 is not necessary
anymore, or maybe it is but can be restricted to the few affected
platforms.

This leaves meta.c and the filemon files to be converted. I didn't do
them since they are not well covered by the unit tests.

make(1): format job.h consistently

make(1): split Job.jobPipe into 2 separate fields

Just because these file descriptors have to be in an array when they are
created is not reason enough to keep this array and a few access macros
in the Job struct. It's better to have separate fields, as they can be
documented independently.

make(1): rewrite documentation of Job

The comment above a struct is supposed to give a high-level overview.
The previous comment didn't do that but instead listed the struct fields
in declaration order in a numbered list, which was not helpful.

make(1): normalize whitespace in source code

There is no more space tab. Either only tabs or only spaces or tabs
followed by spaces, but not spaces followed by tabs.

make(1): remove nonexistent function from header

make(1): fix type of Shell.noPLen

make(1): add specific typedefs for lists

These typedefs are only intended to help human readers, they do not
provide any type-safety. They also make the pointers explicit, which
had been hidden before by the typedef for Lst and LstNode. Typing a few
'*' is less work than finding out which of the many types are pointers
and which aren't.

In meta.c, the variable "ln" served two completely different purposes,
which have been split again. Register allocation is the job of the
compiler, not of the human source code reader.

make(1): remove ReturnStatus, SUCCESS and FAILURE

These are used in so few places now that it is easier to use a simple
Boolean for them.

make(1): remove unused declarations from job.h

make(1): remove form feeds in the code

make(1): follow naming conventions for multiple-inclusion guards

This avoids undefined behavior.

make(1): remove trailing whitespace

If commandShell hasErrCtl is true, set shellErrFlag for use by
CompatRunCommand() so that behavior in jobs and compat mode
remains consistent.

Add a .STALE special target that gets invoked when dependency files contain
stail entries.

branches: 1.40.6; 1.40.12;
Add meta.c which implements "meta" mode for make.
In this mode, a .meta file is created for each target, capturing
the expanded commands used, any command output, and if filemon(9)
is available, a record of system calls which are of interest.
Not enabled unless USE_META=yes is set when building make.
Also, if FILEMON_H exists, meta.c will be compiled to use filemon(9).

Honour the TMPDIR environment variable instead of always using /tmp
as a place to store temporary files.

branches: 1.38.2;
Use NULL instead of -1 cast to the relavant type (usually via NIL).
This was a suggestion from christos - so blame him if there is a deep
reason for using -1 :-)

back all changes out until I fix it properly.

fix typo in comment

- use pid_t/size_t as appropriate instead of int.
- use %ld to print pids.
- fix a bit of lint.
- WARNS=4

Add the ability to tweak the token output before targets in job mode.
Eg.
.MAKE.JOB.PREFIX=${.newline}---${.MAKE:T}[${.MAKE.PID}]
would produce
---make[1234] target ---

branches: 1.33.8;
There is no need to resize the pollfd array, it can only ever have 2+maxJobs
entries, so allocate at startup.
Use an 'int jobPipe[2]' within the job structure, and create pipes directly
into it. Common up the code that creates all the pipes - making them all
non-block on the read side in the process.
Call Job_CatchChildren() directly from Job_CatchOutput() so that it only
gets called when a child actually exits.
NB: Something causes a 'pregnant pause' if (for example) you call 'nbmake obj'
in src/tools. Introduced between netbsd 3 and 4.

Rip out the code for the undocumented -P (don't use pipes for command
output) option. I'm sure it is baggage from the past.

Instead of trying to report all the 'job suspended' and 'job resumed'
messages whan make itself is suspended (ie by ^Z) before make actually
suspends, supress the messages during this sequence.
This means we don't care that they would be output after the suspend
and we can stop attempting to reap child status from withing the signal
handler (which doesn't work for recursive parallel makes).
The code simplification means that we can remove much of the code that
blocked signals - since the signal handlers (expect that for ^C and friends)
now do almost no work.

Complete revamp of the way make handles job control signals.
- Send each type of signal to its own handler.
- Only call JobFinish when a process exits, in particular don't 'fake up'
'exitstatus' for jobs being continued, nor call it for suspends.
- When a job is stopped, use an entire variable to remember the fact, so
we know we need to send a SIGCONT. Don't change any other state.
- In order to report '*** [job3] Suspended' before we suspend ourselves we
have to call waitpid() from the signal handler - where we don't want to
process job termination events. Save the exit status and process later.
The code now handles:
- jobs that suspend themselves
- jobs exiting while suspended
- jobs that don't actually suspend at all
Hoewever it still does printfs() from the signal handler, and I haven't yet
stopped it thrashing the signal mask.

Replace the two lists 'job_list' (previously 'jobs') and 'stoppedJobs'
with a table that is malloced with 'maxJobs' entries.
Add a 'job_state' field to the Job type that exactly follows which of
the old lists the job was on (or not).
Change all the code that scanned the lists to scan the array.
No logic changes in this commit.
(Soon we'll no longer need to lock out signals for the changes to job
statuses that are done from signal handlers now that there is no linked list.)

Fixed the bug reported in PR 33866, which is that the :Q operator does not
handle newlines correctly. Ok'ed by christos.

There is no need to count jobs and job tokens.
If we don't create the job pipe, use the '-j n' option to limit the number
of tokens we will remove from the pipe.

JOB_FIRST is always set (since we only execute the shell once for every
target). Nuke it.

Remove most of the code seemed to be there to support 'remote jobs'.
It isn't clear that it ever worked, if it did it has almost certainly
bitrotted in the last 12 years. I'm not even sure all the required
components were present.
I suspect it was written to attempt to use a 'farm' of diskless sun3s.
In any case the apparant random assignment fo jobs to other systems doesn't
actually seem like a good idea!
Things like 'distcc' han be used to help slow systems run native builds.
Removing this code also simplifies make, and should let me speed up some of
its processing - without worrying about bitrotting it further.

Make .NOTPARALLEL stop the job engine issuing more than one token (instead
of trying to de-jobify the make.
You can now put .NOTPARALLEL in a submake of a recursive make (where it is
using a job-token pipe from the outer make and have it only run a single job.
You can also specify .NOTPARRALEL in the root makefile of a large recursive
make and have the submakes run multiple commands.
Add some diagnostics printfs (enabled with -dp) to the parser.

Expunge last references to jobTokensFree

When a parallel make job completes, always put the job token back into the
job_pipe and collect another one for the next job.
If we are aborting, remove all the 'normal' job tokens and add an 'error' one.
If we get an 'error' token, remove any other tokens, re-insert the error
token and exit (with error 'cos that is easier).
Add the current pid to some of the DEBUG(JOB) traces.
Combined effect is that parallel makes actually stop some fairly shortly
after an error, rather than running on long enough to fill the scrollback.

Fixes for PR#18573 (make -j not stopping correctly on error).
Using -e in sh/ksh to stop on error doesn't work with grouped commands. At
least for any SUSE compliant sh(1). Instead, extend the Shell struct and add
errOut which provides a template to use to check error status from commands.

Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22365, verified by myself.

Allow .SHELL: to control the shell used by compat mode too.
Add a shell spec for ksh - a nice portable posix shell.
Document .SHELL:

Pass WARNS=3

Fixed race condition that would cause make -j to pause for five
seconds if a SIGCHLD arrived while make was not blocked in poll(),
by making the SIGCHLD handler write to a pipe included in the poll.
Avoided the need to implement a duplicate fix for the USE_SELECT case
by emulating poll() in terms of select() when USE_SELECT is defined.
Fixes bin/18895.

Remove !__STDC__ stuff, de-__P(), ANSIfy, and de-register.

A number of semi-related changes.
1. make -dx turns on DEBUG_SHELL which causes sh -x to be used where
possible.
2. PrintOnError() is now called when make is stopping due to an error.
This routine reports the curdir and the value of any variables listed
in MAKE_PRINT_VAR_ON_ERROR.
3. Variables set via command line, are propagated to child-makes via
MAKEFLAGS. This behaviour appears to be necessary for POSIX (according
to the GNU folk anyway).
4. Do not reset MAKEFILE when reading ".depend" as this rather eliminates the
usefulness of ${MAKEFILE}.
5. Added ${.newline} as a simple means of being able to include \n in the
result of a :@ loop expansion.
6. Set ${MAKE_VERSION} if defined. Need to come up with a useful value.

Reviewed: christos

Include token counts in trace output.
Don't let tokensFree go negative.

Avoid token leaks when aborting after an error.

Do lazy token release - don't return tokens to the pipe unless we're
waiting for something or we're done.

Add a token-passing scheme to allow a recursive make to successfully
use -j; all make's in a recursive build cooperate to limit the total
number of jobs, using a token-passing scheme.

The current token passing algorithm is similar to the one implemented
by gmake; there is a single pipe which is inherited through the entire
process hierarchy; tokens are obtained by reading a byte from the
"read end" of the pipe, and are returned by writing them to the "write
end". This exact algorithm is likely to change in the future.

Implementation details:
- Use the new trace facility to allow measurement of the
effectiveness of different token-passing schemes
- Get a token in MakeStartJobs(), return it in Make_Update()
- Eliminate Job_Full() and the jobFull global since they are
redundant with token system.
- Add an "internal" -J option (to pass the token pipe fd's down to
submakes) and a -T option for tracing.
- Change how compatMake is forced so that -j means something when
inherited by submakes.
- When waiting for a token, poll the token-passing pipe as well as
the output pipes of existing jobs.

correct performance regression of recent change from select() to
poll() for parallel make:
- Make the poll() code behave more like the select() code: sleep for
a bit waiting for output rather than busy-wait (eww).
- Install a no-op SIGCHLD handler so that poll/select wake up early
(with -1/EINTR) when a child exits.
- Change the default sleep time from 500ms to 5 seconds since we now
wake up promptly when a child exits.

oops forgot to commit this one.

use 6 X's instead of 5 for portability.

conditionalize prototypes for REMOTE support
(seems that REMOTE support is incomplete anyway)

PR/5210: Hauke Fath: make core dumps with .SHELL
Unfortunately this revealed a deeper problem with the brk_string code.
To fix it:
- remove sharing of the buffer between brk_string invocations
- change the semantics of brk_string so that the argument array
starts with 0, and return the buffer where the strings are
stored

branches: 1.5.2;
- Merge in FreeBSD and Lite2 changes.
- Fix bug where a non-archive target with a .a suffix would always
be considered to be out of date, since it does not have a TOC.

branches: 1.4.6;
- $NetBSD$ rcsids
- Fixed so that .[A-Z]* targets that do not match keywords are ignored as
Posix mandates
- Added .PHONY target keyword

fixes/improvements from Christos Zoulas <christos@deshaw.com>.

Add RCS identifiers.

branches: 1.1.1;
Initial revision