tests/lint: reorganize tests

Tests for a single working language construct don't usually need a
separate file, they can be grouped together, like in expr.c or gcc.c.

tests/lint: remove unused lint1-skip-if selector

lint: test format strings from snprintb calls

The functions snprintb and snprintb_m are specific to NetBSD, and their
format strings are tricky to get correct. Provide some assistance in
catching the most common mistakes.

lint: tab-align message numbers in err.c

By replacing block comments with end-of-line comments, the comments take
up less space and thus no longer require to be indented by 6 spaces.

The messages and their comments are used in 3 places: the manual page
lint.7, the err-msgs.h header for debug mode, and check-msgs.lua to
verify that the comments above the message IDs correspond to the actual
messages.

No functional change.

lint1: remove option -m

The list of lint messages is only needed for generating the manual page
lint.7. Since 2022-07-06, that list is extracted from the source code
instead of compiling and then running lint1.

lint: only generate err-msgs.h in debug mode

lint: use TOOL_SED instead of sed

Use ${TOOL_SED} instead of (the host) sed, as the
latter doesn't necessarily support the non-POSIX -E.

PR bin/57533

lint: fix use-after-free bug in GCC statement expressions

lint: in debug mode, don't generate empty message texts

Most of these messages have an /* unused */ comment in err.c, but some
don't.

lint: enable more lint warnings for its own source code

tests/lint: spell platform identifiers for 'long double' consistently

The test file names don't have a hyphen, so the identifiers shouldn't
have one either.

lint: warn about extern declarations outside headers

https://mail-index.netbsd.org/tech-userlevel/2023/03/15/msg013727.html

lint: split platform-specific test for loss of accuracy

Lint distinguishes between platforms where size_t is unsigned int and
platforms where size_t is unsigned long.

lint: inline separate Makefile

lint: add additional queries that are not enabled by default

In the last 18 months, several lint warnings have been made adjusted to
allow common usage patterns. For example, lint no longer warns about a
constant condition in the statement 'do { ... } while (false)' (message
161), as this pattern is well-known in statement-like macros, making it
unlikely that the 'false' is a mistake. Another example is casts
between unequal pointer types (message 247) for a few well-known
patterns that are unlikely to be bugs.

Occasionally, it is useful to query the code for patterns or events that
would not justify a warning. These patterns are modeled as predefined
queries that can be selected individually, in addition to and
independently of the existing warnings and errors.

New queries can be added as needed, in the same way as new warnings.
Queries that are deemed no longer used can be deactivated in the same
way as warnings that are no longer used.

As long as none of the queries is enabled, they produce a minimal
overhead of querying a single global variable. Computations that are
more expensive than a few machine instructions should be guarded by
any_query_enabled.

https://mail-index.netbsd.org/source-changes-d/2022/06/28/msg013716.html

ok christos@

tests/lint: remove .exp files, as they have become redundant

Now that each lint1 test lists all generated diagnostics as 'expect'
comments, the information from the .exp files is no longer needed. The
only information that gets lost is the order of the diagnostics, which
is mostly relevant for paired messages like 'inconsistent definition' +
'previous definition was here'.

lint: disambiguate sym_t.s_value

Having a single variable for 4 different purposes with different types
makes the code unnecessarily hard to follow.

No functional change.

lint: split CTCONST into BOOL_CONST and ENUM_CONST

Having a unified compile-time constant "storage class" made the code
more difficult to understand.

No functional change.

Fix build of xlint/line1 with OBJDIR set.

lint: do not warn about comparison 'unsigned <= 0'

Seen in scanners generated by Flex, and about 50 occurrences in the
NetBSD src and xsrc tree, all of which are not suspicious of being bugs.

lint: suppress remaining lint warnings in generated scan.c

Warning 162 about 'unsigned <= 0' feels too ambitious, it may be
restricted to the clearly wrong 'unsigned < 0' in the future.

Warnings 192 and 214 are a result of the strict bool check, but the
error messages are suppressed, which makes it hard to see why lint says
the local variable were unused and the function would not return a
value.

Warning 307 about unused static variables is OK for generated code.

lint: generate date of lint.7 from err.c

usr.bin: enable lint checks for most programs in /usr/bin

Previously, lint was only activated for the libraries, but not for the
kernel or userland programs. Activate lint for the programs in /usr/bin
for now, more will follow later.

This only affects builds that set MKLINT=yes.

lint: clean gcov files for lint2 and xlint as well

lint: extract debug logging to separate file

Lint currently has several different kinds of debug log:

* The -DDEBUG log is controlled at compile time.
* The -d command line options enables some other debug logging.
* The -DYYDEBUG log for parsing is controlled at compile time.
* The -y command line option only has an effect in -DYYDEBUG mode.

Extracting the logging into a separate file is a first step towards
unifying these logs and making the code for debug logging stand out less
than the current #ifdef DEBUG.

No functional change.

lint: add debugging output for the grammar tokens

No functional change outside debug mode.

lint: move scl_name from print.c to decl.c

It is only needed there.

tests/lint: when adding a new test, set an invalid option

This forces the test author to think about whether any custom options
are really needed. While here, mention the other possible
customizations as well.

GCC git commit 91f66e78cc141da77ff9e0e3c8519e1af3f26c07[1] introduced
a regression in sh. In addition to the intended change (based on the
commit message), an apparently unintended change was made, inverting a
comparison. This broke sh builds and our workaround (so far) was to
compile xlint/lint1 with -O0.

Revert the comparison to what it was before and remove the -O0 hack
from xlint/lint1.

[1] https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=91f66e78cc141da77ff9e0e3c8519e1af3f26c07

lint: in strict bool mode, allow mixed types in generated C code

This allows flex lexers to be run through lint in strict bool mode.

lint: include scan.c in 'make lint'

This theoretically enables strict bool mode for the few remaining code
in scan.l. Since scan.l is not yet detected as generated code, all
interesting errors have to be suppressed though.

tests/lint: do not modify t_integration on "make add-test"

The test names are no longer listed individually.

apply sh3 vs lex.c fix for GCC >= 9. use shorter idiom.

fixes build issue reported by jdbaker in private email.

ensure defined(HAVE_GCC) before use of ${HAVE_GCC}

lint1: add bug report for internal GCC compiler error

Hack to avoid gcc 9 internal compiler error for sh3.

lint: remove WARNS=3, falling back to the default WARNS=5

It's strange that GCC does not warn about the nonliteral format strings
in lint1/err.c, lint2/msg.c and lint2/read.c, despite -Wformat=2, but
Clang does.

lint: fix strict bool mode errors in cgram.y

The code generated by yacc already adheres to strict bool mode, in
default mode as well as in debug mode.

Running lint on the generated cgram.c as well avoids most of the
"declared but not used" warnings from lint2.

The code generated by lex does not adhere to strict bool mode though.
Suppressing the errors from strict bool mode works, but then lint1 runs
into an assertion failure:

assertion "tn != NULL || nerr != 0" failed
in expr at tree.c:3610 near scan.c:822

This leaves several warnings about "declared but not used" for the
functions from lex.c.

lint1: remove redundant dependency

This is already handled by "make depend".

lint: use distinct struct tags for type_t in lint1 and lint2

Having two similar but still different definitions of 'struct type' is
unnecessarily confusing. Exchange this confusion for 8 lines of
straight-forward preprocessing code.

lint: move check for strict bool mode into separate file

No functional change.

lint: warn about for wrong type cast in argument to ctype.h functions

The argument to most of the functions from <ctype.h> "shall either be
representable as an 'unsigned char' or shall equal the value of the
macro EOF".

When confronted with the infamous warning 'array subscript has type
char', there are enough programmers who don't know the background of
that warning and thus fix it in a wrong way. Neither GCC nor Clang
explain its warning to target these programmers.

Both GCC and Clang warn about 'array subscript has type char', but they
ignore the other requirements of the <ctype.h> functions, even though
these are in the C standard library.

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94182
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=95177
https://stackoverflow.com/a/60696378

lint: add missing space when adding a new test case

lint: make lint's own code pass the strict bool mode

No functional change.

lint: document how to compile lint1 in debug mode

lint: remove gcov results on "make clean"

lint: warn about mismatch in getopt handling

lint: move lexer code from scan.l to lex.c

Previously, the generated scan.c was not included when running "make
lint". Similarly, cgram.c is still not included.

lint: automate adding a test for lint1

lint: rename ops.c to oper.c

The file ops.c had previously been autogenerated. This meant that in a
NetBSD build, it was generated in OBJDIR, and a build that had just
updated src/usr.bin would fail. For a build that last ran on
2020-12-01, and again today, it looks like this:

# link lint1/lint1
cc ... -o lint1 cgram.lo ... ops.lo ... tyname.lo
/usr/bin/ld: ops.lo: in function `initmtab':
ops.c:(.text+0x63): undefined reference to `STRUCT_ASSIGN'

This is caused by ops.c existing in OBJDIR, so the new version in
NETBSDSRCDIR is not looked at. To prevent this, use oper.c instead as
the filename, which has not been used before.

https://mail-index.netbsd.org/source-changes-d/2021/01/09/msg013096.html

lint: clean up generation of the operator tables

Instead of running a shell program that runs an AWK program that
generates the two files ops.c and ops.h, just define the operator tables
once in ops.def and use these definitions flexibly in ops.c and op.h.

lint: in DEBUG mode, verify printf parameters for messages

Since several years GCC validates printf-style strings, and there is no
reason not to let GCC do that work. This prevents bugs like the
segmentation fault that was fixed in tree.c 1.109 from 2021-01-01.

By default, lint is compiled with DEBUG off, but it's easy enough to
compile it in debug mode once in a while.

lint: remove custom LINTFLAGS

These flags are missing the options -g (GNUisms) and -S (C9X mode).
Without these flags, NetBSD's system headers cannot be compiled as they
make heavy use of __inline and __attribute__.

Update LLVM to 10.0.0git (01f3a59fb3e2542fce74c768718f594d0debd0da)

branches: 1.52.16;
build ops.h first, because trying to build dependencies and ops.h at the
same time leads to spurious failures.

add LDEBUG

commit test

cvs test

sort

branches: 1.47.6;
correct debugging flags.

branches: 1.46.4;
generate ops table dynamically. Will move the other table from tree.c there
too.

branches: 1.45.2;
remove the remaining non-ELF code.

remove almost all the ability to build netbsd with an a.out target.
we're ELF now, and there are many missing checks against OBJECT_FMT.
if we ever consider switching, the we can figure out what new ones
we need but for now it's just clutter.

this doesn't remove any of the support for exec_aout or any actually
required-for-boot a.out support, only the ability to build a netbsd
release in a.out format. ie, most of this code has been dead for
over a decade.

i've tested builds on vax, amd64, i386, mac68k, macppc, sparc, atari,
amiga, shark, cats, dreamcast, landisk, mmeye and x68k. this covers
the 5 MACHINE_ARCH's affected, and all the other arch code touched.
it also includes some actual run-time testing of sparc, i386 and
shark, and i performed binary comparison upon amiga and x68k as well.


some minor details relevant:
- move shlib.[ch] from ld.aout_so into ldconfig proper, and cut them
down to only the parts ldconfig needs
- remove various unused source files
- switch amiga bootblocks to using elf2bb.h instead of aout2bb.h

add variable declarations in for loops for c99

Compile lint with WARNS=3.
Currently there are too many -Wsign-compare issues for WARNS=4.

Comment out CFLAGS+=-g.

this code isn't -Wsign-compare friendly -- drop the WARNS to 3.

branches: 1.39.2;
WARNS=4

branches: 1.38.6;
preliminary _Complex support.
NB: Does not really understand type conversions between complex and doubles.

branches: 1.37.16;
Fix c99 initialization issues. Now the regression tests work.

WFORMAT is no more...

Factor out tyname() so that it can be used both by lint1 and lint2.
Since type_t is different between lint1.h and lint2.h include the
appropriate file depending on the pass. Make the argument mismatch
error print the type names of the types involved. Now that we have
a tyname() function we can fix the rest of the pass2 warnings to be
more explanatory, but not now.

Rework how MAKEVERBOSE operates:

* Don't bother prefixing commands with a line of ${_MKCMD}\
and instead rely upon "make -s". This is less intrusive on
all the Makefiles than the former. Idea from David Laight.

* Rename the variables use to print messages. The scheme now is:
_MKMSG_FOO Run _MKMSG 'foo'
_MKTARGET_FOO Run _MKMSG_FOO ${.TARGET}
From discussion with Alistair Crooks.

support MAKEVERBOSE and use ${HOST_SH}

Rework how dependency generation is performed:

* DPSRCS contains extra dependencies, but is _NOT_ added to CLEANFILES.
This is a change of behaviour. If a Makefile wants the clean semantics
it must specifically append to CLEANFILES.
Resolves PR toolchain/5204.

* To recap: .d (depend) files are generated for all files in SRCS and DPSRCS
that have a suffix of: .c .m .s .S .C .cc .cpp .cxx

* If YHEADER is set, automatically add the .y->.h to DPSRCS & CLEANFILES

* Ensure that ${OBJS} ${POBJS} ${LOBJS} ${SOBJS} *.d depend upon ${DPSRCS}

* Deprecate the (short lived) DEPENDSRCS


Update the various Makefiles to these new semantics; generally either
adding to CLEANFILES (because DPSRCS doesn't do that anymore), or replacing
specific .o dependencies with DPSRCS entries.

Tested with "make -j 8 distribution" and "make distribution".

Back out debugging Makefile.

PR/21706: Krister Walfridsson: CVT nodes not handled in ?: expressions.

handle void ({ })

Now that <bsd.prog.mk> DTRT if HOSTPROG is defined (i.e, it is a no-op),
there's no need to special-case .include-ing it.

remove dependencies on ${LIBC} ${LIBM} and ${LIBTERMCAP} static libraries
from host tool targets. Commits fix from PR toolchain/21495

need -lm in the HOSTPROG case too...

now needs -lm for finite(3)

Roll in fixes to permit cross-compiling from non-NetBSD hosts. This
round has been tested on Solaris/x86 and Linux hosts.

* Add host tools cap_mkdb, ctags, m4, uudecode.
* Protect __RCSID() and __COPYRIGHT() better.
* Reduce the number of places that need to include "config.h", to keep
sources closer to their "vanilla" versions.
* Add more compat #defines and autoconf-checked functions.

Centralize the initialization/declaration of the ttab.

If USETOOLS=yes, as in a cross build or standard tool-based build, create
lint1.7 using the host lint1 tool. Otherwise run lint1 from here, as has
been done historically.

Should fix toolchain/15001.

Major overhaul of src/tools and host toolchain. Summary of changes:

* Rewrite src/tools Make logic to work like the rest of the tree wrt
"dependall" and "install". The old "make build" hack is gone.

* Remove the MKTOOLS logic. This was linked to the "make build" hack,
and was only needed because TOOLDIR originally had no writable default.

* Redo the GNU configure/make logic to make it fit reasonably in a
BSD make wrapper. Use new ${.ALLTARGETS} variable to scan for
targets in $(srcdir), and mark them with .MADE: to prevent rebuilding.

* Only build cross tools in src/tools; remove some messy logic in
src/usr.* and src/gnu/usr.* that would do target filename rewriting
(improves consistency and readability).

* Add the ability to build cross gdb at tool build time by setting
MKCROSSGDB (default no) to "yes" in mk.conf.

* Add src/tools/groff and set up paths to work with this cross groff.

Give host tool builds better control over the HOST_CPPFLAGS.

Add support for putting prefixes on the name of lint(1)'s front
and back ends. This means that it can be built as a cross tool.
Part of the fix for bin/14280.

Back out part of rev. 1.16 (my build didn't pick up the -I from
../Makefile.inc the first time around, for some reason.)

Set a CPP define on the command line to indicate which object
format lint(1) is being targeted at, and use this knowledge
as appropriate in the target param headers.

Let lint.7 get auto-built by the bsd.man.mk logic (don't explicitly depend
on it via "realall:"). Fixes lint.7 problem in src/tools reported by mrg.

While here, actually -I the arch subdir to pull in the correct targparam.h.

Move CPPFLAGS addition for lint1/arch to Makefile.inc so each subdir will
pick them up

Split out target parameters into separate header files, and
pull the correct one in based on the MACHINE_CPU variable.
MACHINE_CPU will be set according to the target system we are
building for by <bsd.own.mk>.

One component of addressing bin/14280.

Add hooks to allow toolchain bits to be reachover-built at the top level.

More format string cleanup by sommerfeld.

Fix problem building lint.7.

better way to handle lint.7

build lint.7 man page

Make gcc 2.96 (and maybe earlier) happier. Include <stdlib.h>,<string.>,
etc. as appropriate to get exit,srncmp,abs,abort,etc.
Add -I${.CURDIR} to a few Makefiles

branches: 1.7.2; 1.7.10;
convert from NOxxx= to MKxxx=no.
include <bsd.own.mk> if testing a MKxxx variable.

.y.c <sys.mk> rule fixes. Don't create a y.tab.h file unless asked for,
and use smarter creation of the header file.

use CPPFLAGS instead of CFLAGS

branches: 1.4.2;
Remove hack for (very old) GCC bug on i386.

hack around a bug in our gcc.

RCS id cleanup

branches: 1.1.1;
Initial revision