ctype(3): New environment variable LIBC_ALLOWCTYPEABUSE.

If set, this does not force the ctype(3) functions to crash when
passed invalid inputs -- instead, they will return nonsense results,
and possibly print warnings to stderr, as is their right in
implementing undefined behaviour.

The nature of the nonsense results is unspecified. Currently, is*()
will always return true (even if that leads to mutually contradictory
conclusions, like isalpha and isdigit, or isgraph and isblank), and
tolower/toupper() will always return EOF. But perhaps in the future
the results may be randomized.

This way, if an application like firefox crashes on ctype abuse, you
can opt to accept the consequences of nonsense results instead by
running `env LIBC_ALLOWCTYPEABUSE= firefox' until the application is
fixed.

PR lib/58208: ctype(3) provides poor runtime feedback of abuse

branches: 1.17.2;
ctype(3): Simplify definitions of ctype/tolower/toupper tables.

Clarify comment while here.

No functional change intended. No change to `readelf -a' output on
amd64 or aarch64.

PR lib/58208: ctype(3) provides poor runtime feedback of abuse

libc: Restore ELF symbol sizes for _C_ctype_tab_ &c.

This is needed for dynamic position-dependent executables that refer
directly to _C_type_tab_ to get correct copy relocations to see the
table content.

Unfortunately, such executables won't get a guard page.

Fortunately, referring to _C_ctype_tab_ directly (and not the
indirection _ctype_tab_ as the ctype(3) macros do) is very weird and
unlikely to happen in the real world (none of the public interfaces
use it; it is exported for libc++.so/libstdc++.so to use, but those
aren't pies). So missing the guard page in this case is probably not
so bad.

The symbol sizes are also needed for, e.g., gdb to nicely identify
addresses that lie in the table.

PR lib/58208: ctype(3) provides poor runtime feedback of abuse

ctype(3): Put guard pages before the C ctype/tolower/toupper tables.

This also only affects machines where char is signed for now. (But
maybe it would be worth doing unconditionally; users could still try
to pass in explicit `signed char' inputs.)

PR lib/58208: ctype(3) provides poor runtime feedback of abuse

branches: 1.14.40;
Use more consistent naming for tolower/toupper translation table for the
C locale. From Takehiko Nozaki with changes to use __BUILD_LEGACY.

branches: 1.13.6; 1.13.12;
more split ctype.h -> sys/ctype_inline.h, sys/ctype_bits.h

1. hide _CTYPE_PRIVATE section in ctype.h, move them to private header ctype_local.h.
2. do not use _CTYPE_NUM_CHARS macro to read data from LC_CTYPE(old BSDCTYPE style) database.
because 1<<CHAR_BIT is MD, so i added MI macro _CTYPE_CACHE_SIZE(1<<8).
3. remove _NB_CACHED_RUNE macro, use _CTYPE_CACHE_SIZE instead.

merge christos-time_t

Fixes PR lib/39662, shortcomings in LC_{MONETARY,NUMERIC,TIME,MESSAGES} db format.
ok'ed by core and releng.
(thanks for agc@, snj@ and i'm sorry for long time patience).

[libc]
- localeio.[ch] and lc*.[ch] in src/lib/libc/locale was replaced by
new locale-db implementation using citrus_db backend,
see src/lib/libc/citrus/citrus_lc_*.[ch].
- add citrus_bcs_strtou?l.c. don't use strtou?l locale implementation
internally, because they're locale-aware function.
- add some stubs for multi-locale issue, see {current,global}_locale.c.
- remove some obsolete file, setrunelocale.c, ___runetype_mb.c.
- remove __savectype() from ctypeio.[ch].

[tools]
- mklocale(1): add new option ``-t'' that generates new style
LC_{MONETARY,NUMERIC,TIME,MESSAGES} locale-db format.
- chrtbl(1): added ctypeio.[ch] for __savectype().

[locale-db]
- added en_US.US-ASCII locale.
- removed some shareable locale definition file:
en_US.US-ASCII -> en_US.ISO8859-1, en_US.UTF-8
zh_CN.eucCN -> zh_CN.GB18030
and more...see src/share/locale/*/Makefile.
- remove obsoleted locale sr_YU, added new locale sr_ME, sr_RS.
- change locale name ja_JP.ISO2022-JP* -> ja_JP.ISO-2022-JP*
for X11's locale.alias file alignments.
- fix regression test, wrong wcs?width(3), NAN/INF usage.

i tested release-build following arch:
i386, amd64, hpc{mips,arm,sh}, sparc64, vax.

citrus_lc_*.[ch] also can read old-plain-text style locale-db.
so that backward compatibility is keeped, but lc*.[ch] can't read
new citrus_db'ed locale-db and localeio.c never check sanity,
so forward compatibility is broken ;-<

old mklocale(1) doesn't know -t option, so you have to rebuild toolchain.

branches: 1.9.38; 1.9.40;
netbsd.org->NetBSD.org

Fix RCSID's

Winning Strategies has placed this code in the public domain.

merge with 4.4-Lite, keeping local changes. clean up Ids

Fix spelling error in Copyright notice

Explicitly declare function return types to silence "gcc -Wall".

Separate the toupper() and tolower() functions so the tables *really* don't get
included unless you need them.

Declare rcsid strings so they are stored in text segment.

Moved toupper and tolower tables from ctype_.c to their own files --- I
received complaints about using shorts in the table (but i need a range
of -1..255), so now the tables will not be used unless either toupper()
or tolower() (and soon, setlocale()) are used. This can save up to 514
bytes.

In toupper_.c and tolower_.c make sure that our assumption of EOF == -1
holds.

Fixed bug where _toupper_tab_ was initialized pointing to _C_tolower_tab.