Home | History | Annotate | Download | only in locale
History log of /src/lib/libc/locale/Makefile.inc
RevisionDateAuthorComments
 1.69  15-Sep-2025  riastradh ctype(3): New environment variable LIBC_ALLOWCTYPEABUSE.

If set, this does not force the ctype(3) functions to crash when
passed invalid inputs -- instead, they will return nonsense results,
and possibly print warnings to stderr, as is their right in
implementing undefined behaviour.

The nature of the nonsense results is unspecified. Currently, is*()
will always return true (even if that leads to mutually contradictory
conclusions, like isalpha and isdigit, or isgraph and isblank), and
tolower/toupper() will always return EOF. But perhaps in the future
the results may be randomized.

This way, if an application like firefox crashes on ctype abuse, you
can opt to accept the consequences of nonsense results instead by
running `env LIBC_ALLOWCTYPEABUSE= firefox' until the application is
fixed.

PR lib/58208: ctype(3) provides poor runtime feedback of abuse
 1.68  09-Sep-2024  riastradh branches: 1.68.2;
mbsnrtowcs(3), wcsnrtombs(3): Add man pages for these mouthfuls.

Based on patch drafted by abhinav@, with a little additional tidying
by me.

PR standards/52343: wcsnrtombs missing man-page
 1.67  15-Aug-2024  riastradh libc: New functions c8rtomb(3) and mbrtoc8(3).

New in C23, for converting from UTF-8 to locale-dependent multibyte
sequences (c8rtomb) or vice versa (mbrtoc8), along with the new type
char8_t.

Conditional on either:
- _NETBSD_SOURCE
- _ISOC23_SOURCE
- __STDC_VERSION__ >= 202311L

(Riding the libc minor bump from this morning for the UTF-16/UTF-32
versions from C11.)

PR standards/58601: uchar.h C23 compliance: char8_t, mbrtoc8, c8rtomb
 1.66  15-Aug-2024  riastradh libc: New C11 functions mbrtoc16, mbrtoc32, c16rtomb, c32rtomb.

The mbrtoc16/32 functions read mulitbyte strings according to the
current locale into UTF-16/32 code unit sequences; the c16/32rtomb
functions write UTF-16/32 code unit sequences into multibyte strings
according to the current locale. The `r' means restartable: they
work incrementally and pick up where they left off.

NOTE: This bumps the libc minor version, since it adds new symbols.

PR lib/52374: <uchar.h> missing
 1.65  15-Feb-2021  christos branches: 1.65.6; 1.65.8;
Add missing man pages (from FreeBSD)
 1.64  18-Aug-2013  joerg branches: 1.64.20; 1.64.28;
Rename fixup to update_global to better reflect the purpose.
Remove the various Current pointers, they are implementation internals
that shouldn't be leaked. Remove _CATEGORY_DEFAULT related handling and
directly pick the implemenation pointer from the C locale.
Merge the C locale data into global_locale.c with the exception of
_DefaultRuneLocale. Mark that one hidden. Move _RUNE_LOCALE and
_CITRUS_CTYPE into multibyte.h and pick up the correct rl_citrus_ctype
for non-default locales.
 1.63  17-May-2013  joerg Provide explicit LC_C_LOCALE accessor and drop the various NULL checks.
Provide LC_GLOBAL_LOCALE in a way that works with all locale functions.
Merge constant data used by the initial global locale and the C locale.
Drop function call layer for _current_locale() and directly return the
locale reference, not a pointer to it. Use protected access for global
variables, so that libc references can avoid the GOT overhead.
 1.62  30-Apr-2013  joerg Make that to process PATH_LOCALE the first time _find_category is
called. Use this to implement newlocale and provide duplocale/freelocale
as well. Based on patches by Takehiko Nozaki with simplications and fix
for the init order by myself.
 1.61  14-Apr-2013  joerg Provide a const copy of global_locale for libc-internal use.
This will be used by *_l when a NULL pointer is given.
 1.60  20-Jan-2012  joerg branches: 1.60.6;
Remove code for CITRUS!=yes.
 1.59  14-Aug-2011  christos branches: 1.59.2;
annotate that some functions use non-literal format strings.
 1.58  19-Jun-2010  tnozaki 1. refactoring new locale-db(RuneCT10) loading method with mmap(2).
2. remove unused field from _RuneLocale.
3. localeio(CITRUS=no) can read new locale-db(RuneCT10) now.
 1.57  07-Jun-2010  tnozaki 1. MB_LEN_MAX switch MD to MI.
2. unfortunately hppa's MB_LEN_MAX is defined incorrectly 6 instead of 32
so we have to add more setlocale(3) __RENAME func, __setlocale50.
3. move setlocale1.c and setlocale32.c to lib/libc/compat/locale/*
prepareing for next libc major crunk.
4. bump libc minor version.
 1.56  01-Jun-2010  tnozaki refactoring old locale-db(BSDCTYPE) loading method with mmap(2).
 1.55  30-May-2010  tnozaki 1. fix build problem CITRUS=no
2. make CITRUS={yes,no} knob friendly with libc/compat.
 1.54  11-Jan-2009  christos merge christos-time_t
 1.53  02-Jan-2009  tnozaki Fixes PR lib/39662, shortcomings in LC_{MONETARY,NUMERIC,TIME,MESSAGES} db format.
ok'ed by core and releng.
(thanks for agc@, snj@ and i'm sorry for long time patience).

[libc]
- localeio.[ch] and lc*.[ch] in src/lib/libc/locale was replaced by
new locale-db implementation using citrus_db backend,
see src/lib/libc/citrus/citrus_lc_*.[ch].
- add citrus_bcs_strtou?l.c. don't use strtou?l locale implementation
internally, because they're locale-aware function.
- add some stubs for multi-locale issue, see {current,global}_locale.c.
- remove some obsolete file, setrunelocale.c, ___runetype_mb.c.
- remove __savectype() from ctypeio.[ch].

[tools]
- mklocale(1): add new option ``-t'' that generates new style
LC_{MONETARY,NUMERIC,TIME,MESSAGES} locale-db format.
- chrtbl(1): added ctypeio.[ch] for __savectype().

[locale-db]
- added en_US.US-ASCII locale.
- removed some shareable locale definition file:
en_US.US-ASCII -> en_US.ISO8859-1, en_US.UTF-8
zh_CN.eucCN -> zh_CN.GB18030
and more...see src/share/locale/*/Makefile.
- remove obsoleted locale sr_YU, added new locale sr_ME, sr_RS.
- change locale name ja_JP.ISO2022-JP* -> ja_JP.ISO-2022-JP*
for X11's locale.alias file alignments.
- fix regression test, wrong wcs?width(3), NAN/INF usage.

i tested release-build following arch:
i386, amd64, hpc{mips,arm,sh}, sparc64, vax.

citrus_lc_*.[ch] also can read old-plain-text style locale-db.
so that backward compatibility is keeped, but lc*.[ch] can't read
new citrus_db'ed locale-db and localeio.c never check sanity,
so forward compatibility is broken ;-<

old mklocale(1) doesn't know -t option, so you have to rebuild toolchain.
 1.52  17-May-2008  ginsbach branches: 1.52.4; 1.52.6;
Add support for additional locale categories: LC_MESSAGES, LC_MONETARY,
LC_NUMERIC.

The code used to load LC_TIME was refactored in to a more general routine.
This common routine is now used to load LC_TIME along with the newly added
categories.

Changes discussed with/reviewed by christos.
 1.51  28-Mar-2007  manu branches: 1.51.10; 1.51.12;
Add support for LC_TIME, from Joachim Kuebart, through PR lib/10877
 1.50  13-Oct-2006  tnozaki PR/30809 added manpages of wcswcs(3) and wcscoll(3), wcsxfrm(3)
(wcs?width and wprintf are already in repository).
 1.49  03-Jun-2006  tnozaki fix lib/33476:
added following manpages (derived from FreeBSD),
wcwidth(3) wcswidth(3)
wcstol(3) wcstoll(3) wcstoimax(3)
wcstoul(3) wcstoull(3) wcstoumax(3)
and update doc/TODO.i18n.
 1.48  15-Apr-2006  tnozaki add c99 wcstof(3) and wcstold(3).
requested by skrll AT netbsd DOT org, thanks.

bump libc minor 138 -> 139.
 1.47  30-Mar-2005  wiz Add man page for wcsftime from FreeBSD.
 1.46  30-Mar-2005  christos PR/29826: J. T. Conklin: Add wcsftime(3) (from FreeBSD)
 1.45  11-Mar-2003  tshiozak branches: 1.45.6;
add wcstoll, wcstoimax, wcstoull and wcstoumax. (lib/20656)
- these are defined in C99 (wcsto[ui]max) and SUSv3.
- these implementations are written by sigsegv@s25.xrea.com,
derived from NetBSD's wcstol/wcstoul.
- I just added some comments.
 1.44  05-Mar-2003  tshiozak changes related to lib/20576 and lib/20577.
- add wcstok() and wcswcs(),
- add wcstok(3),
- move btowc() and wctob() into each ctype modules,
- bump ctype interface ABI version,
- bump i18n modules' minor and
- update sets.
and fix a stupid bug in _RUNE_ISCACHED macro.
 1.43  04-Mar-2003  tshiozak add manual pages for iswctype(), towctrans(), wctype() and wctrans().
 1.42  03-Mar-2003  tshiozak - add btowc.3 / wctob.3.
- fix an unappropriate comment in wctob.c
 1.41  03-Mar-2003  tshiozak - add wctob() function. (I forgot it at the last commit.)
- fix btowc(); I quite misunderstood about it.
 1.40  02-Mar-2003  tshiozak add some ISO C 1995 I18N functions and types:
btowc, wctrans, towctrans, wcscoll, wcsxfrm, wctype_t and wctrans_t.
 1.39  18-Mar-2002  tshiozak add manpages for mb*/wc* functions.
 1.38  17-Mar-2002  tshiozak refine i18n stuffs.
- add libc/citrus directory.
this directory contains the common stuffs not only for locale
but also other facilities relating to i18n (e.g. iconv).
- To keep the binary compatibility for the ctype modules easily,
the interface between libc and the modules is simplified.
- For the future integrated extension, module pool is renamed
from "/usr/lib/runemodule" to "/usr/lib/i18n".
In the future, this directory will contain the modules for "iconv",
"collation", etc.
- some cosmetic changes.
- Bug fix for runetype.h; __attribute__((__packed__)) is placed at
the wrong position and it is invalid unintentionally.
But, the all members of the structures seem well-aligned. Thus,
this bug causes no problem, hopefully.

HEADS UP:
- /usr/lib/rumemodule is obsoleted. If you use the multibyte locales,
you need to install /usr/lib/i18n/* from sys/lib/i18n_module .
- The binary compatibility of /usr/share/locale/*/LC_CTYPE is probably kept.
Perhaps, the bug fix about __packed__ mentioned above breaks the
compatibility... Be careful especially on 64bit platforms.
 1.37  13-Feb-2002  yamt add a function to handle alias names.
(primarily for nls, encoding and locale names)
 1.36  27-Sep-2001  yamt add wcstod/wcstol/wcstoul.
 1.35  01-Apr-2001  tshiozak Remove UTF-2. UTF-2 is superceded by UTF-8.
see also:
http://mail-index.netbsd.org/tech-userlevel/2001/03/30/0000.html
http://mail-index.netbsd.org/tech-userlevel/2001/03/30/0001.html
http://mail-index.netbsd.org/tech-userlevel/2001/03/30/0002.html
 1.34  25-Jan-2001  itojun branches: 1.34.2;
(finally!) enable multibyte LC_CTYPE locale support.

the functionality will be available for dynamic-linked binary only, due to
limitation with dlopen() from within statically-linked binaries.
(statically-linked binary can still enjoy singlebyte LC_CTYPE locale support)

from citrus project.
 1.33  25-Jan-2001  itojun split setlocale.c for smaller footprint (do not pull things in too much).
have magic number check in locale modules.
rename: lib/libc/locale/setlocale_sb.c -> setlocale1.c

NOTE: this commit does not enable multibyte locale support, yet.
 1.32  22-Jan-2001  itojun move __mb_cur_max to dedicated *.o, so that we do not pull in
locale stuff just by calling printf().
 1.31  27-Dec-2000  itojun re-enable citrus (rune)locale support.

XXX please clean and recompile libc if you get this revision. UPDATE=yes
may leave obsolete *.o.
 1.30  25-Dec-2000  itojun byte default, use (non-citrus) singlebyte locale. if we compile
ld.elf_so with citrus locale, it bombs.
XXX this is shortterm workaround. if you are willing to test citrus locale,
use RUNE=yes in libc/locale/Makefile.inc.
 1.29  23-Dec-2000  itojun move wcs* and wmem* from lib/libc/locale to lib/libc/string.
 1.28  22-Dec-2000  itojun add wcsl{cat,cpy} - outside of ISO/IEC 9899:1999,
but i believe it should be there.
 1.27  22-Dec-2000  itojun ad manpage for wide character string manipulation (wmem*/wcs*)
 1.26  22-Dec-2000  itojun manpage for tow*.
 1.25  22-Dec-2000  itojun manpage for isw*.
 1.24  21-Dec-2000  itojun add multibyte locale converters (just for use by testers at this moment,
to be built as dynamically loadable binary)
 1.23  21-Dec-2000  itojun support multibytes in isw*/tow*.
 1.22  21-Dec-2000  itojun first import of citrus LC_CTYPE locale. singlebyte only at this moment.
shot-term TODO:
- enable multibyte locales. this includes loadable locale modules
(#ifdef DLRUNE).
- populate LC_CTYPE definition files.
- other functions such as iconv()
- make libc functions more friendly with multibyte.
- multilingualize other LC_xx
- torture tests

XXX don't forget to "make depend" when you rebuild libc, multibyte.c is
moved from lib/libc/stdlib to lib/libc/locale.
 1.21  21-Dec-2000  itojun add dummy wcwidth() and wcswidth(). wcwidth() will be updated later.
 1.20  21-Dec-2000  itojun have dummy wctype.h and isw*/tow*. part of libc minor 68 -> 69 crank.
equivalent to is*/to* at this moment. we'll put wchar_t code in the
near future.
 1.19  21-Dec-2000  itojun add wcs* functions. consier it as part of libc minor crank 68 -> 69.
add decl of WEOF into wchar.h. ISO/IEC 9899:1999(E)..
 1.18  20-Dec-2000  itojun add wmem* functions, from citrus. bump libc shlib minor.
 1.17  08-Aug-2000  tshiozak Preparation for the future introduction of multibyte locale.
- MB_LEN_MAX is increased to 32.
- To ensure binary compatibility for old executables
under multibyte locale, versioned setlocale is added.
- __mb_len_cur definision is added in setlocale.c
and enable it in stdlib.h .
It is also important for multibyte locale stuffs,
but I just forgot.
 1.16  24-Feb-1999  drochner branches: 1.16.8; 1.16.10;
use ${ARCHDIR} instead of ${.CURDIR}/arch/${MACHINE_ARCH} where appropriate
libc builds on pmax again
 1.15  15-Jul-1997  kleink Link localeconv(3) to setlocale(3).
 1.14  12-Jul-1997  perry add setlocale.3 man page
 1.13  02-Jun-1997  kleink Add support for localized character sets (a.k.a. LC_CTYPE).

Thanks go to Matthias Scheler <tron@lyssa.owl.de> for contributing his initial
work in PR/3592, and to Christos Zoulas for refining it!
 1.12  29-Apr-1997  kleink RCSid police.
 1.11  22-Dec-1994  cgd specify man pages the new way.
 1.10  29-Sep-1994  jtc Minor tweaks to prepare for checkin of real i18n code.
 1.9  16-Jul-1994  jtc branches: 1.9.2;
Added a nl_langinfo(3) man page.
 1.8  21-Jun-1994  jtc Add nl_langinfo(), a SysV-X/Open function that provides access to some
of the locale info.

Some of the POSIX.2 utilities require this information, and I thought it
was better to use an existing interface rather than rolling my own.
 1.7  26-May-1994  deraadt fix typo
 1.6  25-May-1994  jtc New localeconv() implementation that constructs it's return value from
the current numeric and current monetary locale data.
 1.5  25-May-1994  jtc Compile the default locale data for LC_NUMERIC, LC_MONETARY, and LC_TIME.
 1.4  04-Dec-1993  cgd update for .S and MACHINE_ARCH
 1.3  07-Oct-1993  cgd make an arch directory in libc, and move all machine dirs into it
 1.2  01-Aug-1993  mycroft Add RCS identifiers.
 1.1  21-Mar-1993  cgd branches: 1.1.1;
Initial revision
 1.1.1.1  21-Mar-1993  cgd initial import of 386bsd-0.1 sources
 1.9.2.2  16-Jul-1994  jtc Added a nl_langinfo(3) man page.
 1.9.2.1  16-Jul-1994  jtc file Makefile.inc was added on branch netbsd-1-0 on 1994-07-16 05:12:27 +0000
 1.16.10.1  09-Aug-2000  tshiozak pull up the following changes (approved by thorpej):
> cvs rdiff -r1.9 -r1.10 basesrc/include/locale.h
> cvs rdiff -r1.45 -r1.46 basesrc/include/stdlib.h
> cvs rdiff -r1.16 -r1.17 basesrc/lib/libc/locale/Makefile.inc
> cvs rdiff -r1.17 -r1.18 basesrc/lib/libc/locale/setlocale.c
> cvs rdiff -r0 -r1.2 basesrc/lib/libc/locale/setlocale_sb.c
> cvs rdiff -r1.6 -r1.7 syssrc/sys/arch/alpha/include/limits.h
> cvs rdiff -r1.1 -r1.2 syssrc/sys/arch/arm26/include/limits.h
> cvs rdiff -r1.7 -r1.8 syssrc/sys/arch/arm32/include/limits.h
> cvs rdiff -r1.14 -r1.15 syssrc/sys/arch/i386/include/limits.h
> cvs rdiff -r1.12 -r1.13 syssrc/sys/arch/m68k/include/limits.h
> cvs rdiff -r1.13 -r1.14 syssrc/sys/arch/mips/include/limits.h
> cvs rdiff -r1.10 -r1.11 syssrc/sys/arch/pc532/include/limits.h
> cvs rdiff -r1.6 -r1.7 syssrc/sys/arch/powerpc/include/limits.h
> cvs rdiff -r1.2 -r1.3 syssrc/sys/arch/sh3/include/limits.h
> cvs rdiff -r1.11 -r1.12 syssrc/sys/arch/sparc/include/limits.h
> cvs rdiff -r1.7 -r1.8 syssrc/sys/arch/sparc64/include/limits.h
> cvs rdiff -r1.9 -r1.10 syssrc/sys/arch/vax/include/limits.h
>
> Outline:
>
> Preparation for the future introduction of multibyte locale.
> - MB_LEN_MAX is increased to 32.
> - To ensure binary compatibility for old executables
> under multibyte locale, versioned setlocale is added.
> - __mb_len_cur definision is added in setlocale.c
> and enable it in stdlib.h .
> It is also important for multibyte locale stuffs,
> but I just forgot.
 1.16.8.1  28-May-2000  minoura Citrus Project XPG4DL, an implementation of I18N (locale) framework,
is imported.
 1.34.2.3  22-Mar-2002  nathanw Catch up to -current.
 1.34.2.2  08-Mar-2002  nathanw Catch up to -current.
 1.34.2.1  08-Oct-2001  nathanw Catch up to -current.
 1.45.6.1  06-Apr-2005  tron Pull up revision 1.46 (requested by christos in ticket #102):
PR/29826: J. T. Conklin: Add wcsftime(3) (from FreeBSD)
 1.51.12.1  23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.51.10.1  18-May-2008  yamt sync with head.
 1.52.6.2  04-Jan-2009  christos merge with head.
 1.52.6.1  17-May-2008  christos file Makefile.inc was added on branch christos-time_t on 2009-01-04 17:02:19 +0000
 1.52.4.1  15-Jan-2009  snj Pull up following revision(s) (requested by tnozaki in ticket #222):
distrib/sets/lists/base/mi: revision 1.788
distrib/utils/libhack/Makefile.inc: revision 1.23
distrib/utils/libhack/Makefile: revision 1.22
etc/mtree/NetBSD.dist: revision 1.387
include/locale.h: revision 1.15
lib/libc/citrus/Makefile.inc: revision 1.6
lib/libc/citrus/citrus_aliasname_local.h: revision 1.1
lib/libc/citrus/citrus_bcs.h: revision 1.5
lib/libc/citrus/citrus_bcs_strtol.c: revision 1.1
lib/libc/citrus/citrus_bcs_strtoul.c: revision 1.1
lib/libc/citrus/citrus_csmapper.c: revision 1.9
lib/libc/citrus/citrus_fix_grouping.h: revision 1.1
lib/libc/citrus/citrus_lc_ctype.c: revision 1.1
lib/libc/citrus/citrus_lc_messages.c: revision 1.1
lib/libc/citrus/citrus_lc_messages.h: revision 1.1
lib/libc/citrus/citrus_lc_monetary.c: revision 1.1
lib/libc/citrus/citrus_lc_monetary.h: revision 1.1
lib/libc/citrus/citrus_lc_numeric.c: revision 1.1
lib/libc/citrus/citrus_lc_numeric.h: revision 1.1
lib/libc/citrus/citrus_lc_template.h: revision 1.1
lib/libc/citrus/citrus_lc_template_decl.h: revision 1.1
lib/libc/citrus/citrus_lc_time.c: revision 1.1
lib/libc/citrus/citrus_lc_time.h: revision 1.1
lib/libc/citrus/citrus_module.c: revision 1.8
lib/libc/citrus/citrus_namespace.h: revision 1.7
lib/libc/citrus/modules/citrus_euc.c: revision 1.13
lib/libc/gen/Makefile.inc: revision 1.166
lib/libc/gen/isctype.c: revision 1.17 via patch
lib/libc/gen/tolower_.c: revision 1.10
lib/libc/gen/toupper_.c: revision 1.10
lib/libc/iconv/Makefile.inc: revision 1.3
lib/libc/iconv/iconv.c: revision 1.7
lib/libc/locale/Makefile.inc: revision 1.53
lib/libc/locale/___runetype_mb.c: file removal
lib/libc/locale/_wctrans.c: revision 1.7
lib/libc/locale/_wctrans_local.h: revision 1.3
lib/libc/locale/_wctype.c: revision 1.1
lib/libc/locale/_wctype_local.h: revision 1.1
lib/libc/locale/aliasname.c: revision 1.3
lib/libc/locale/aliasname_local.h: revision 1.2
lib/libc/locale/bsdctype.c: revision 1.1
lib/libc/locale/bsdctype.h: revision 1.1
lib/libc/locale/ctypeio.c: revision 1.8
lib/libc/locale/ctypeio.h: revision 1.2
lib/libc/locale/current_locale.c: revision 1.1
lib/libc/locale/dummy_lc_collate.c: revision 1.1
lib/libc/locale/dummy_lc_template.h: revision 1.1
lib/libc/locale/fix_grouping.c: revision 1.1
lib/libc/locale/fix_grouping.h: revision 1.1
lib/libc/locale/generic_lc_all.c: revision 1.1
lib/libc/locale/generic_lc_template.h: revision 1.1
lib/libc/locale/generic_lc_template_decl.h: revision 1.1
lib/libc/locale/global_locale.c: revision 1.1
lib/libc/locale/iswctype.c: file removal
lib/libc/locale/iswctype_mb.c: revision 1.1
lib/libc/locale/iswctype_sb.c: revision 1.6
lib/libc/locale/lcmessages.c: file removal
lib/libc/locale/lcmessages.h: file removal
lib/libc/locale/lcmonetary.c: file removal
lib/libc/locale/lcmonetary.h: file removal
lib/libc/locale/lcnumeric.c: file removal
lib/libc/locale/lcnumeric.h: file removal
lib/libc/locale/lctime.c: file removal
lib/libc/locale/lctime.h: file removal
lib/libc/locale/localeconv.c: revision 1.15
lib/libc/locale/localeio.c: revision 1.2
lib/libc/locale/localeio.h: revision 1.2
lib/libc/locale/localeio_lc_ctype.c: revision 1.1
lib/libc/locale/localeio_lc_messages.c: revision 1.1
lib/libc/locale/localeio_lc_monetary.c: revision 1.1
lib/libc/locale/localeio_lc_numeric.c: revision 1.1
lib/libc/locale/localeio_lc_time.c: revision 1.1
lib/libc/locale/multibyte.h: revision 1.4
lib/libc/locale/multibyte_amd1.c: revision 1.6
lib/libc/locale/multibyte_c90.c: revision 1.5
lib/libc/locale/nb_lc_messages_misc.h: revision 1.1
lib/libc/locale/nb_lc_monetary_misc.h: revision 1.1
lib/libc/locale/nb_lc_numeric_misc.h: revision 1.1
lib/libc/locale/nb_lc_template.h: revision 1.1
lib/libc/locale/nb_lc_template_decl.h: revision 1.1
lib/libc/locale/nb_lc_time_misc.h: revision 1.1
lib/libc/locale/nl_langinfo.c: revision 1.12
lib/libc/locale/rune.c: revision 1.31
lib/libc/locale/rune.h: revision 1.13
lib/libc/locale/rune_local.h: revision 1.11
lib/libc/locale/runeglue.c: revision 1.14
lib/libc/locale/runetable.c: revision 1.18
lib/libc/locale/setlocale.c: revision 1.55 via patch
lib/libc/locale/setlocale_local.h: revision 1.1
lib/libc/locale/setrunelocale.c: file removal
lib/libc/nls/Makefile.inc: revision 1.9
lib/libc/nls/catopen.c: revision 1.26
lib/libc/string/Makefile.inc: revision 1.68
lib/libc/string/wcscmp.c: revision 1.6
lib/libc/string/wcsncmp.c: revision 1.6
lib/libc/string/wcswidth.c: file removal
lib/libc/string/wmemcmp.c: revision 1.4
regress/lib/libc/locale/Makefile: revision 1.4
regress/lib/libc/locale/ctype1/Makefile: revision 1.4
regress/lib/libc/locale/ctype1/en_US.UTF-8.exp.uue: revision 1.2
regress/lib/libc/locale/ctype1/ja_JP.ISO-2022-JP.exp.uue: revision 1.1
regress/lib/libc/locale/ctype1/ja_JP.ISO-2022-JP.in.uue: revision 1.1
regress/lib/libc/locale/ctype1/ja_JP.ISO2022-JP.exp.uue: file removal
regress/lib/libc/locale/ctype1/ja_JP.ISO2022-JP.in.uue: file removal
regress/lib/libc/locale/ctype1/ja_JP.SJIS.exp.uue: revision 1.2
regress/lib/libc/locale/ctype1/ja_JP.eucJP.exp.uue: revision 1.2
regress/lib/libc/locale/ctype2/Makefile: revision 1.5
regress/lib/libc/locale/ctype2/ja_JP.ISO-2022-JP-2.in.uue: revision 1.1
regress/lib/libc/locale/ctype2/ja_JP.ISO2022-JP2.in.uue: file removal
regress/lib/libc/locale/ctype3/Makefile: revision 1.5
regress/lib/libc/locale/mbtowc/Makefile: revision 1.3
regress/lib/libc/locale/mbtowc/ja_JP.ISO-2022-JP: revision 1.1
regress/lib/libc/locale/mbtowc/ja_JP.ISO2022-JP: file removal
regress/lib/libc/locale/wcstod/wcstod_test.c: revision 1.2
share/locale/Makefile.locale: revision 1.1
share/locale/Makefile: revision 1.5
share/locale/ctype/Makefile: revision 1.28
share/locale/locale.alias: revision 1.11
share/locale/messages/Makefile: revision 1.5
share/locale/messages/en_US.ISO8859-1.src: file removal
share/locale/messages/en_US.US-ASCII.src: revision 1.1
share/locale/messages/ja_JP.ISO-2022-JP.src: revision 1.1
share/locale/messages/ja_JP.ct.src: revision 1.1
share/locale/messages/sr_ME.ISO8859-2.src: revision 1.1
share/locale/messages/sr_ME.ISO8859-5.src: revision 1.1
share/locale/messages/sr_YU.ISO8859-2.src: file removal
share/locale/messages/sr_YU.ISO8859-5.src: file removal
share/locale/messages/sr_YU.UTF-8.src: file removal
share/locale/messages/zh_CN.GB18030.src: file removal
share/locale/messages/zh_TW.eucTW.src: revision 1.1
share/locale/monetary/Makefile: revision 1.5
share/locale/monetary/af_ZA.ISO8859-1.src: revision 1.2
share/locale/monetary/am_ET.UTF-8.src: revision 1.2
share/locale/monetary/be_BY.CP1131.src: file removal
share/locale/monetary/be_BY.CP1251.src: revision 1.2
share/locale/monetary/be_BY.ISO8859-5.src: revision 1.2
share/locale/monetary/be_BY.UTF-8.src: revision 1.2
share/locale/monetary/bg_BG.CP1251.src: revision 1.2
share/locale/monetary/bg_BG.UTF-8.src: revision 1.2
share/locale/monetary/ca_ES.ISO8859-1.src: revision 1.2
share/locale/monetary/cs_CZ.ISO8859-2.src: revision 1.2
share/locale/monetary/cs_CZ.UTF-8.src: revision 1.2
share/locale/monetary/da_DK.ISO8859-1.src: revision 1.2
share/locale/monetary/de_AT.ISO8859-1.src: revision 1.2
share/locale/monetary/de_CH.ISO8859-1.src: revision 1.2
share/locale/monetary/de_DE.ISO8859-1.src: revision 1.2
share/locale/monetary/el_GR.ISO8859-7.src: revision 1.2
share/locale/monetary/en_AU.ISO8859-1.src: revision 1.2
share/locale/monetary/en_CA.ISO8859-1.src: revision 1.2
share/locale/monetary/en_GB.ISO8859-1.src: revision 1.2
share/locale/monetary/en_GB.UTF-8.src: revision 1.2
share/locale/monetary/en_IE.UTF-8.src: revision 1.2
share/locale/monetary/en_NZ.ISO8859-1.src: revision 1.2
share/locale/monetary/en_US.ISO8859-1.src: revision 1.2
share/locale/monetary/en_US.US-ASCII.src: revision 1.1
share/locale/monetary/es_ES.ISO8859-1.src: revision 1.2
share/locale/monetary/et_EE.ISO8859-15.src: revision 1.2
share/locale/monetary/fi_FI.ISO8859-1.src: revision 1.2
share/locale/monetary/fr_BE.ISO8859-1.src: revision 1.2
share/locale/monetary/fr_CA.ISO8859-1.src: revision 1.2
share/locale/monetary/fr_FR.ISO8859-1.src: revision 1.2
share/locale/monetary/he_IL.UTF-8.src: revision 1.2
share/locale/monetary/hi_IN.ISCII-DEV.src: revision 1.2
share/locale/monetary/hr_HR.ISO8859-2.src: revision 1.2
share/locale/monetary/hu_HU.ISO8859-2.src: revision 1.2
share/locale/monetary/hy_AM.ARMSCII-8.src: revision 1.2
share/locale/monetary/hy_AM.UTF-8.src: revision 1.2
share/locale/monetary/is_IS.ISO8859-1.src: revision 1.2
share/locale/monetary/it_IT.ISO8859-1.src: revision 1.2
share/locale/monetary/ja_JP.ISO-2022-JP.src: revision 1.1
share/locale/monetary/ja_JP.UTF-8.src: revision 1.2
share/locale/monetary/ja_JP.eucJP.src: revision 1.2
share/locale/monetary/kk_KZ.PT154.src: revision 1.2
share/locale/monetary/kk_KZ.UTF-8.src: revision 1.2
share/locale/monetary/ko_KR.UTF-8.src: revision 1.2
share/locale/monetary/ko_KR.eucKR.src: revision 1.2
share/locale/monetary/lt_LT.ISO8859-13.src: revision 1.2
share/locale/monetary/mn_MN.UTF-8.src: revision 1.2
share/locale/monetary/nl_BE.ISO8859-1.src: revision 1.2
share/locale/monetary/nl_NL.ISO8859-1.src: revision 1.2
share/locale/monetary/no_NO.ISO8859-1.src: revision 1.2
share/locale/monetary/pl_PL.ISO8859-2.src: revision 1.2
share/locale/monetary/pl_PL.UTF-8.src: revision 1.2
share/locale/monetary/pt_BR.ISO8859-1.src: revision 1.2
share/locale/monetary/pt_PT.ISO8859-1.src: revision 1.2
share/locale/monetary/ro_RO.ISO8859-2.src: revision 1.2
share/locale/monetary/ru_RU.CP1251.src: revision 1.2
share/locale/monetary/ru_RU.CP866.src: revision 1.2
share/locale/monetary/ru_RU.ISO8859-5.src: revision 1.2
share/locale/monetary/ru_RU.KOI8-R.src: revision 1.2
share/locale/monetary/ru_RU.UTF-8.src: revision 1.2
share/locale/monetary/sk_SK.ISO8859-2.src: revision 1.2
share/locale/monetary/sl_SI.ISO8859-2.src: revision 1.2
share/locale/monetary/sr_ME.ISO8859-2.src: revision 1.1
share/locale/monetary/sr_RS.ISO8859-2.src: revision 1.1
share/locale/monetary/sr_YU.ISO8859-2.src: revision 1.2
share/locale/monetary/sr_YU.ISO8859-5.src: revision 1.2
share/locale/monetary/sr_YU.UTF-8.src: revision 1.2
share/locale/monetary/sv_SE.ISO8859-1.src: revision 1.2
share/locale/monetary/tr_TR.ISO8859-9.src: revision 1.2
share/locale/monetary/uk_UA.CP1251.src: revision 1.2
share/locale/monetary/uk_UA.ISO8859-5.src: revision 1.2
share/locale/monetary/uk_UA.KOI8-U.src: revision 1.2
share/locale/monetary/uk_UA.UTF-8.src: revision 1.2
share/locale/monetary/zh_CN.GB18030.src: revision 1.2
share/locale/monetary/zh_CN.UTF-8.src: revision 1.2
share/locale/monetary/zh_CN.eucCN.src: revision 1.2
share/locale/monetary/zh_HK.Big5hkscs.src: revision 1.1
share/locale/monetary/zh_HK.UTF-8.src: revision 1.2
share/locale/monetary/zh_TW.Big5.src: revision 1.2
share/locale/numeric/Makefile: revision 1.4
share/locale/numeric/am_ET.UTF-8.src: file removal
share/locale/numeric/en_US.ISO8859-1.src: file removal
share/locale/numeric/en_US.US-ASCII.src: revision 1.1
share/locale/numeric/ja_JP.eucJP.src: file removal
share/locale/numeric/ko_KR.eucKR.src: file removal
share/locale/numeric/mn_MN.UTF-8.src: file removal
share/locale/numeric/sr_ME.ISO8859-2.src: revision 1.1
share/locale/numeric/sr_YU.ISO8859-2.src: file removal
share/locale/numeric/sr_YU.ISO8859-5.src: file removal
share/locale/numeric/zh_CN.eucCN.src: file removal
share/locale/time/Makefile: revision 1.5
share/locale/time/en_US.ISO8859-1.src: file removal
share/locale/time/en_US.US-ASCII.src: revision 1.1
share/locale/time/ja_JP.ISO-2022-JP.src: revision 1.1
share/locale/time/ja_JP.ct.src: revision 1.1
share/locale/time/sr_ME.ISO8859-2.src: revision 1.1
share/locale/time/sr_ME.ISO8859-5.src: revision 1.1
share/locale/time/sr_ME.UTF-8.src: revision 1.1
share/locale/time/sr_YU.ISO8859-2.src: file removal
share/locale/time/sr_YU.ISO8859-5.src: file removal
share/locale/time/sr_YU.UTF-8.src: file removal
share/locale/time/zh_CN.GB18030.src: file removal
share/locale/time/zh_TW.eucTW.src: revision 1.1
usr.bin/locale/locale.c: revision 1.6
usr.bin/mklocale/Makefile: revision 1.12
usr.bin/mklocale/lex.l: revision 1.14
usr.bin/mklocale/mklocaledb.c: revision 1.1
usr.bin/mklocale/yacc.y: revision 1.25
usr.sbin/chrtbl/Makefile: revision 1.8
usr.sbin/chrtbl/ctypeio.c: revision 1.1
usr.sbin/chrtbl/ctypeio.h: revision 1.1
Fixes PR lib/39662, shortcomings in LC_{MONETARY,NUMERIC,TIME,MESSAGES}
db format.
ok'ed by core and releng.
(thanks for agc@, snj@ and i'm sorry for long time patience).
[libc]
- localeio.[ch] and lc*.[ch] in src/lib/libc/locale was replaced by
new locale-db implementation using citrus_db backend,
see src/lib/libc/citrus/citrus_lc_*.[ch].
- add citrus_bcs_strtou?l.c. don't use strtou?l locale implementation
internally, because they're locale-aware function.
- add some stubs for multi-locale issue, see {current,global}_locale.c.
- remove some obsolete file, setrunelocale.c, ___runetype_mb.c.
- remove __savectype() from ctypeio.[ch].
[tools]
- mklocale(1): add new option ``-t'' that generates new style
LC_{MONETARY,NUMERIC,TIME,MESSAGES} locale-db format.
- chrtbl(1): added ctypeio.[ch] for __savectype().
[locale-db]
- added en_US.US-ASCII locale.
- removed some shareable locale definition file:
en_US.US-ASCII -> en_US.ISO8859-1, en_US.UTF-8
zh_CN.eucCN -> zh_CN.GB18030
and more...see src/share/locale/*/Makefile.
- remove obsoleted locale sr_YU, added new locale sr_ME, sr_RS.
- change locale name ja_JP.ISO2022-JP* -> ja_JP.ISO-2022-JP*
for X11's locale.alias file alignments.
- fix regression test, wrong wcs?width(3), NAN/INF usage.

i tested release-build following arch:
i386, amd64, hpc{mips,arm,sh}, sparc64, vax.

citrus_lc_*.[ch] also can read old-plain-text style locale-db.
so that backward compatibility is keeped, but lc*.[ch] can't read
new citrus_db'ed locale-db and localeio.c never check sanity,
so forward compatibility is broken ;-<
old mklocale(1) doesn't know -t option, so you have to rebuild toolchain.
 1.59.2.2  22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.59.2.1  17-Apr-2012  yamt sync with head
 1.60.6.2  20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.60.6.1  23-Jun-2013  tls resync from head
 1.64.28.1  11-Oct-2024  martin Pull up following revision(s) (requested by riastradh in ticket #1897):

lib/libc/locale/mbsrtowcs.3: revision 1.14
lib/libc/locale/mbsrtowcs.3: revision 1.15
lib/libc/locale/wcsrtombs.3: revision 1.14
lib/libc/locale/wcsrtombs.3: revision 1.15
lib/libc/locale/wcsrtombs.3: revision 1.16
distrib/sets/lists/comp/mi: revision 1.2473
lib/libc/locale/Makefile.inc: revision 1.68
lib/libc/locale/mbsrtowcs.3: revision 1.13

mbsnrtowcs(3), wcsnrtombs(3): Add man pages for these mouthfuls.

Based on patch drafted by abhinav@, with a little additional tidying
by me.

PR standards/52343: wcsnrtombs missing man-page

mbsnrtowcs(3), wcsnrtombs(3): brush ip markup

SYNOPSIS fixes, uppercase NUL and NULL
 1.64.20.2  21-Jul-2017  perseant Move Unicode <-> ku/ten mapping into the individual codec modules.
Mapping is based on existing iconv data for single-byte encodings,
and included for several, but not all, multibyte encodings.
 1.64.20.1  14-Jul-2017  perseant Initial commit of a mostly-working implementation of __STDC_ISO_10646__,
with collation support using the Unicode Collation Algorithm.

The conversion from men/ku/ten form to Unicode is a gross hack at present.
Fixing this, and fleshing out the LC_COLLATE locale component, are next
on the agenda.
 1.65.8.1  02-Aug-2025  perseant Sync with HEAD
 1.65.6.2  14-Oct-2024  martin Pull up following revision(s) (requested by riastradh in ticket #976):

lib/libc/locale/c32rtomb.3: revision 1.10
lib/libc/locale/c32rtomb.3: revision 1.9
lib/libc/locale/c32rtomb.3: revision 1.11
tests/lib/libc/locale/t_mbrtoc32.c: revision 1.1
distrib/sets/lists/base/shl.mi: revision 1.988
lib/libc/include/namespace.h: revision 1.204
lib/libc/include/namespace.h: revision 1.205
lib/libc/locale/mbrtoc16.3: revision 1.1
lib/libc/locale/mbrtoc16.c: revision 1.1
lib/libc/locale/mbrtoc16.3: revision 1.2
lib/libc/locale/mbrtoc16.c: revision 1.2
lib/libc/locale/mbrtoc16.3: revision 1.3
lib/libc/locale/mbrtoc16.c: revision 1.3
lib/libc/locale/mbrtoc32.3: revision 1.1
lib/libc/locale/mbrtoc32.c: revision 1.1
tests/lib/libc/locale/t_c16rtomb.c: revision 1.1
lib/libc/locale/mbrtoc32.c: revision 1.2
lib/libc/locale/mbrtoc16.3: revision 1.4
lib/libc/locale/mbrtoc16.c: revision 1.4
lib/libc/locale/mbrtoc32.3: revision 1.2
tests/lib/libc/locale/t_c16rtomb.c: revision 1.2
lib/libc/locale/mbrtoc32.c: revision 1.3
lib/libc/locale/mbrtoc16.3: revision 1.5
lib/libc/locale/mbrtoc16.c: revision 1.5
lib/libc/locale/mbrtoc32.3: revision 1.3
tests/lib/libc/locale/t_c16rtomb.c: revision 1.3
lib/libc/locale/mbrtoc32.c: revision 1.4
lib/libc/locale/mbrtoc16.3: revision 1.6
lib/libc/locale/mbrtoc16.c: revision 1.6
lib/libc/locale/mbrtoc32.3: revision 1.4
tests/lib/libc/locale/t_c16rtomb.c: revision 1.4
lib/libc/locale/mbrtoc32.c: revision 1.5
lib/libc/locale/mbrtoc16.3: revision 1.7
lib/libc/locale/mbrtoc16.c: revision 1.7
lib/libc/locale/mbrtoc32.3: revision 1.5
tests/lib/libc/locale/t_c16rtomb.c: revision 1.5
lib/libc/locale/mbrtoc32.c: revision 1.6
lib/libc/locale/mbrtoc16.3: revision 1.8
lib/libc/locale/mbrtoc32.3: revision 1.6
tests/lib/libc/locale/t_c16rtomb.c: revision 1.6
lib/libc/locale/mbrtoc32.c: revision 1.7
lib/libc/locale/mbrtoc16.3: revision 1.9
lib/libc/locale/mbrtoc32.3: revision 1.7
lib/libc/locale/mbrtoc32.c: revision 1.8
lib/libc/locale/mbrtoc32.3: revision 1.8
lib/libc/locale/mbrtoc32.c: revision 1.9
distrib/sets/lists/comp/mi: revision 1.2468
lib/libc/locale/mbrtoc32.3: revision 1.9
distrib/sets/lists/comp/mi: revision 1.2469
lib/libc/locale/c32rtomb.h: revision 1.1
lib/libc/locale/c32rtomb.h: revision 1.2
include/Makefile: revision 1.147
share/man/man3/uchar.3: revision 1.1
share/man/man3/uchar.3: revision 1.2
tests/lib/libc/locale/t_c32rtomb.c: revision 1.1
distrib/sets/lists/comp/mi: revision 1.2470
lib/libc/locale/c16rtomb.3: revision 1.1
lib/libc/locale/c16rtomb.c: revision 1.1
lib/libc/locale/c16rtomb.3: revision 1.2
lib/libc/locale/c16rtomb.c: revision 1.2
lib/libc/locale/c16rtomb.3: revision 1.3
lib/libc/locale/c16rtomb.c: revision 1.3
lib/libc/locale/c16rtomb.3: revision 1.4
lib/libc/locale/c16rtomb.c: revision 1.4
lib/libc/locale/c16rtomb.3: revision 1.5
lib/libc/locale/c16rtomb.c: revision 1.5
lib/libc/locale/c16rtomb.3: revision 1.6
lib/libc/locale/c16rtomb.c: revision 1.6
lib/libc/locale/c16rtomb.3: revision 1.7
lib/libc/locale/c16rtomb.c: revision 1.7
lib/libc/locale/c16rtomb.3: revision 1.8
lib/libc/locale/c16rtomb.3: revision 1.9
distrib/sets/lists/tests/mi: revision 1.1330
distrib/sets/lists/tests/mi: revision 1.1331
distrib/sets/lists/tests/mi: revision 1.1332
tests/lib/libc/locale/t_uchar.c: revision 1.1
tests/lib/libc/locale/t_uchar.c: revision 1.2
tests/lib/libc/locale/t_uchar.c: revision 1.3
tests/lib/libc/locale/t_mbrtoc16.c: revision 1.1
tests/lib/libc/locale/t_mbrtoc16.c: revision 1.2
tests/lib/libc/locale/t_mbrtoc16.c: revision 1.3
include/uchar.h: revision 1.1
include/uchar.h: revision 1.2
include/uchar.h: revision 1.3
include/uchar.h: revision 1.4
include/uchar.h: revision 1.5
tests/lib/libc/locale/t_c8rtomb.c: revision 1.1
include/uchar.h: revision 1.6
tests/lib/libc/locale/t_c8rtomb.c: revision 1.2
tests/lib/libc/locale/t_c8rtomb.c: revision 1.3
tests/lib/libc/locale/t_c8rtomb.c: revision 1.4
share/man/man3/Makefile: revision 1.93
tests/lib/libc/locale/t_c8rtomb.c: revision 1.5
tests/lib/libc/locale/t_c8rtomb.c: revision 1.6
tests/lib/libc/locale/t_c8rtomb.c: revision 1.7
lib/libc/shlib_version: revision 1.297
lib/libc/locale/c16rtomb.3: revision 1.10
lib/libc/locale/c16rtomb.3: revision 1.11
tests/lib/libc/locale/t_mbrtoc8.c: revision 1.1
tests/lib/libc/locale/t_mbrtoc8.c: revision 1.2
tests/lib/libc/locale/t_mbrtoc8.c: revision 1.3
lib/libc/locale/mbrtoc16.3: revision 1.10
tests/lib/libc/locale/Makefile: revision 1.15
tests/lib/libc/locale/Makefile: revision 1.16
tests/lib/libc/locale/Makefile: revision 1.17
tests/lib/libc/locale/Makefile: revision 1.18
distrib/sets/lists/debug/mi: revision 1.442
distrib/sets/lists/debug/mi: revision 1.443
distrib/sets/lists/debug/mi: revision 1.444
lib/libc/locale/c8rtomb.3: revision 1.1
lib/libc/locale/c8rtomb.c: revision 1.1
lib/libc/locale/c8rtomb.3: revision 1.2
lib/libc/locale/c8rtomb.c: revision 1.2
lib/libc/locale/c8rtomb.3: revision 1.3
lib/libc/locale/c8rtomb.c: revision 1.3
lib/libc/locale/c8rtomb.3: revision 1.4
lib/libc/locale/c8rtomb.c: revision 1.4
lib/libc/locale/c8rtomb.3: revision 1.5
lib/libc/locale/c8rtomb.c: revision 1.5
lib/libc/locale/c8rtomb.3: revision 1.6
lib/libc/locale/c8rtomb.c: revision 1.6
lib/libc/locale/c8rtomb.3: revision 1.7
lib/libc/locale/c8rtomb.3: revision 1.8
lib/libc/locale/c8rtomb.3: revision 1.9
lib/libc/locale/mbrtoc32.h: revision 1.1
lib/libc/locale/mbrtoc32.h: revision 1.2
lib/libc/locale/mbrtoc8.c: revision 1.1
lib/libc/locale/mbrtoc8.3: revision 1.1
lib/libc/locale/mbrtoc8.c: revision 1.2
lib/libc/locale/mbrtoc8.3: revision 1.2
lib/libc/locale/mbrtoc8.c: revision 1.3
lib/libc/locale/mbrtoc8.3: revision 1.3
lib/libc/locale/mbrtoc8.c: revision 1.4
lib/libc/locale/mbrtoc8.3: revision 1.4
lib/libc/locale/Makefile.inc: revision 1.66
lib/libc/locale/mbrtoc8.c: revision 1.5
lib/libc/locale/mbrtoc8.3: revision 1.5
lib/libc/locale/Makefile.inc: revision 1.67
lib/libc/locale/mbrtoc8.c: revision 1.6
lib/libc/locale/mbrtoc8.3: revision 1.6
lib/libc/locale/mbrtoc8.c: revision 1.7
lib/libc/locale/mbrtoc8.3: revision 1.7
lib/libc/locale/mbrtoc8.c: revision 1.8
lib/libc/locale/c32rtomb.3: revision 1.1
lib/libc/locale/c32rtomb.c: revision 1.1
lib/libc/locale/c32rtomb.3: revision 1.2
lib/libc/locale/c32rtomb.c: revision 1.2
lib/libc/locale/c32rtomb.3: revision 1.3
lib/libc/locale/c32rtomb.c: revision 1.3
lib/libc/locale/c32rtomb.3: revision 1.4
lib/libc/locale/c32rtomb.c: revision 1.4
lib/libc/locale/c32rtomb.3: revision 1.5
lib/libc/locale/c32rtomb.c: revision 1.5
lib/libc/locale/c32rtomb.3: revision 1.6
lib/libc/locale/c32rtomb.c: revision 1.6
lib/libc/locale/c32rtomb.3: revision 1.7
lib/libc/locale/c32rtomb.3: revision 1.8

(all via patch)


tests/lib/libc/locale/Makefile: Sort.
No functional change intended.
Preparation for PR lib/52374.

uchar.h: New header file for C11 (and C++11) compliance.

Implementation of the new functions mbrtoc16, c16rtomb, mbrtoc32, and
c32rtomb to come later. Updates for C23 to come later.
PR lib/52374: <uchar.h> missing

libc: New C11 functions mbrtoc16, mbrtoc32, c16rtomb, c32rtomb.

The mbrtoc16/32 functions read mulitbyte strings according to the
current locale into UTF-16/32 code unit sequences; the c16/32rtomb
functions write UTF-16/32 code unit sequences into multibyte strings
according to the current locale. The `r' means restartable: they
work incrementally and pick up where they left off.

NOTE: This bumps the libc minor version, since it adds new symbols.

PR lib/52374: <uchar.h> missing
mbrtoc16(3), mbrtoc32(3): Fix \n in man page examples.
Need to write \en to pacify roff.
PR lib/52374: <uchar.h> missing

c16rtomb(3), c32rtomb(3): Fix more \n in man pages.
Also, tighten an assertion: we left room for a NUL byte at the end.
PR lib/52374: <uchar.h> missing

libc: Use the more idiomatic alignof from stdalign.h.
No functional change intended.
PR lib/52374: <uchar.h> missing

mbrtoc16(3): Simplify surrogate state test.

Turn the finer-grained test into an assertion.
No semantic change intended: we are supposed to control this state,
and we always arrange it this way. (But in principle this could
change the behaviour of buggy programs that violate the mbstate_t
abstraction.)
PR lib/52374: <uchar.h> missing

libc: New functions c8rtomb(3) and mbrtoc8(3).

New in C23, for converting from UTF-8 to locale-dependent multibyte
sequences (c8rtomb) or vice versa (mbrtoc8), along with the new type
char8_t.

Conditional on either:
- _NETBSD_SOURCE
- _ISOC23_SOURCE
- __STDC_VERSION__ >= 202311L
(Riding the libc minor bump from this morning for the UTF-16/UTF-32
versions from C11.)

PR standards/58601: uchar.h C23 compliance: char8_t, mbrtoc8, c8rtomb
libc: c32rtomb and mbrtoc32 are used internally, so weak-alias them.
PR lib/52374: <uchar.h> missing
c8rtomb(3), mbrtoc8(3): Use namespace.h to get private aliases.

This way applications defining the symbols c32rtomb or mbrtoc32 won't
clobber our private definitions, which are slightly more constrained
about their use of mbstate_t than is obvious from the interface
contract.

PR standards/58601: uchar.h C23 compliance: char8_t, mbrtoc8, c8rtomb
mbrtoc16(3), mbrtoc32(3): brush up markup

Split long .Fn lines into Fo/Fa/Fc. Dont indent the list of return
values. Don't use artisanal -width.

Untabify code examples - indented literal displays don't have correct
tab stops consistent with tab stops in the fixed font code, so the
lines end up misaligned in the PostScript output.

c16rtomb(3), c32rtomb(3): brush up markup

mbrtoc16(3), mbrtoc32(3): Simplify return value language.
Also expand BMP only once.
PR lib/52374: <uchar.h> missing

mbrtoc16(3), mbrtoc32(3): No state overlap with mbrtoc8 or c8rtomb.
PR standards/58601: uchar.h C23 compliance: char8_t, mbrtoc8, c8rtomb

mbrtoc32(3): Clarify control flow.
No need for another goto here; let's keep it clearly structured with
a single `out' label.
No functional change intended.
PR lib/52374: <uchar.h> missing

c8rtomb(3), mbrtoc8(3): brush up markup

mbrtoc8(3): Simplify return value language.
PR standards/58601: uchar.h C23 compliance: char8_t, mbrtoc8, c8rtomb

c16rtomb(3), c32rtomb(3): Specify what happens if ps is null.
PR lib/52374: <uchar.h> missing

c8rtomb(3): Specify what happens when ps is null.
PR standards/58601: uchar.h C23 compliance: char8_t, mbrtoc8, c8rtomb

c16rtomb(3), c32rtomb(3): No state overlap with mbrtoc8 or c8rtomb.
PR standards/58601: uchar.h C23 compliance: char8_t, mbrtoc8, c8rtomb

mbrtoc16(3), mbrtoc32(3): Work on deturgidifying prose.
Still maybe not great but at least there's less jargon in most of the
text, without really losing any content.
PR lib/52374: <uchar.h> missing

mbrtoc8(3): Work on deturgidifying prose.
PR standards/58601: uchar.h C23 compliance: char8_t, mbrtoc8, c8rtomb

mbrtoc16(3), mbrtoc32(3): Restore word accidentally removed.
PR lib/52374: <uchar.h> missing

mbrtoc8(3): Restore word accidentally removed.
PR standards/58601: uchar.h C23 compliance: char8_t, mbrtoc8, c8rtomb

c8rtomb(3): Fix possible error descriptions.
The argument c8 can't be a surrogate code point itself (they're in
the range [0xd800,0xdfff], beyond 8-bit values), but the bits of a
surrogate code point could be forced into the UTF-8 format, which is
also invalid.
PR standards/58601: uchar.h C23 compliance: char8_t, mbrtoc8, c8rtomb

c16rtomb(3), c32rtomb(3): Attempt a deturgidification pass.
Limit the jargon around surrogates.
PR lib/52374: <uchar.h> missing

c8rtomb(3): Clarify prose and fix example in caveat.
PR standards/58601: uchar.h C23 compliance: char8_t, mbrtoc8, c8rtomb
c16rtomb(3), c32rtomb(3), mbrtoc16(3), mbrtoc32(3): xref c8 versions.
PR standards/58601: uchar.h C23 compliance: char8_t, mbrtoc8, c8rtomb

mbrtoc16(3): Clarify how many bytes are consumed in special cases.
Fix overlap in RETURN VALUES section.
PR lib/52374: <uchar.h> missing

mbrtoc8(3): Clarify how many bytes are consumed in special cases.
Fix overlap in RETURN VALUES section.
PR standards/58601: uchar.h C23 compliance: char8_t, mbrtoc8, c8rtomb

pass lint, XXX see lint bug.

libc: Add _l variants of the cNrtomb and mbrtocN functions.
These accept an explicit locale parameter, rather than using the
current locale.
Visible under _NETBSD_SOURCE, not exposed otherwise.
NOTE: This adds libc symbols. Riding the libc minor bump for the
non-_l variants of these from two days ago -- hope that's not pushing
it too far.
PR lib/58613: c*rtomb, mbrtoc* should have locale-parametric _l
variants

c8rtomb(3), c16rtomb(3): Add tests for incomplete NUL termination.
PR lib/58615: incomplete c8rtomb, c16rtomb handles NUL termination
wrong

c8rtomb(3), c16rtomb(3): Fix NUL handling.
PR lib/58615: incomplete c8rtomb, c16rtomb handles NUL termination
wrong

c8rtomb(3), c16rtomb(3), c32rtomb(3): Test stateful shift sequences.
PR lib/58612: c8rtomb/c16rtomb/c32rtomb yield suboptimal shift
sequences

c8rtomb(3): Fix digit error in shift sequence test.
PR lib/58612: c8rtomb/c16rtomb/c32rtomb yield suboptimal shift
sequences

c8rtomb(3): Nix __CTASSERT after case label.
I put this in to make it (machine-verifiably) clear that zeroing the
state is the same as returning to the initial conversion state, as
the standard requires, but this is causing build trouble (and will
likely cause more trouble if pulled up) because some definitions of
__CTASSERT make a declaration which is forbidden after a label, so
let's remove it.
PR standards/58601: uchar.h C23 compliance: char8_t, mbrtoc8, c8rtomb

mbrtoc8(3): Fix pasto in comment at top.
No functional change intended.
PR standards/58601: uchar.h C23 compliance: char8_t, mbrtoc8, c8rtomb

mbrtoc8: remove lint-specific workarounds
No binary change.

mbrtoc8: fix comments

mbrtoc16, mbrtoc32: fix comments, remove lint-specific workarounds
No binary change.
t_c8rtomb, t_c16rtomb: Simplify comment.
ESC $ B is technically rather the JIS X 0208-1983 shift sequence, but
since I don't see any way to provoke the JIS X 0208-1978 shift
sequence to come flying out of this conversion (ESC $ @), and I'm not
sure there's any difference in the interpretation, let's just say JIS
X 0208.
PR lib/58612: c8rtomb/c16rtomb/c32rtomb yield suboptimal shift
sequences

c32rtomb(3): Use conversion state to handle shift sequences.
For conversion of Unicode scalar values to coding systems requiring
shift sequences, such as ISO-2022-JP, _citrus_iconv_convert will
always produce:
1. a shift sequence from the initial state to some nondefault state,
like from US-ASCII to JIS X 0208
2. the encoding of the desired characater
3. a shift sequence restoring the initial state
This is unnecessary if the output is already in the state needed to
encoded the desired character. For example, this method produces
seven bytes to encode each YEN SIGN in ISO-2022-JP -- and fourteen,
to encode two consecutive ones -- even though the shift sequence is
only three bytes long and once shifted YEN SIGN takes only one byte.
Instead, convert the Unicode scalar value to a locale-dependent wide
character and encode that, by composing
- _citrus_iconv_convert
=> gives us a multibyte encoding of the character from the initial
state (and restoring the initial state afterward)
- mbrtowc with initial conversion state
=> gives us the single wide character representation
XXX If combining characters are possible here, this may fail.
- wcrtomb with caller's conversion tsate
=> gives us a state-dependent multibyte encoding of the character
XXX Is there a cheaper way to convert from Unicode scalar value to
locale-dependent wide character? It is not obvious to me from the
largely undocumented Citrus machinery, but it would obviously be
better than this somewhat circuitous Rube Goldberg contraption of
chained multibyte APIs.
PR lib/58612: c8rtomb/c16rtomb/c32rtomb yield suboptimal shift
sequences

mbrtoc8(3), mbrtoc16(3): Test consuming shift sequences with state.
This has the side effect of testing mbrtoc32(3) because they are both
defined in terms of it.
PR lib/58618: mbrtocN(3) fails to keep shift state

c8rtomb(3), c16rtomb(3), c32rtomb(3): Suggest MB_LEN_MAX in example.
This way it avoids variable-length arrays, by always allocating the
maximum space that could be occupied by MB_CUR_MAX.

mbrtoc32(3): Use conversion state to handle shift sequences.
PR lib/58618: mbrtocN(3) fails to keep shift state

mbrtoc32(3): Fix name and type of mbrtowc_l return value.
This was from `int mbtowc_l(...)' in an earlier draft and I didn't
update it to size_t when I changed the draft to mbrtowc_l. Caught by
lint.
`mb_len' avoids (harmless) clash with standard C function mblen(3).
PR lib/58618: mbrtocN(3) fails to keep shift state

c32rtomb(3): Fix type of wcrtomb_l return value.
This was from `int wctomb_l(...)' in an earlier draft and I didn't
update it to size_t when I changed the draft to wcrtomb_l. Caught by
lint.
`wc_len' mirrors `mb_len' in the complementary code in mbrtoc32(3) to
avoid clash with standard C function mblen(3).
PR lib/58612: c8rtomb/c16rtomb/c32rtomb yield suboptimal shift
sequences

c8rtomb(3), c16rtomb(3), c32rtomb(3): Attempt to simplify language.

c8rtomb(3), c16rtomb(3), c32rtomb(3): Fix null string output case.
This ignores c8/c16/c32, produces no output anywhere, and just resets
ps to the initial conversion state.
Also just use 0 in the example, not '\0' or L'\0'. This works for
C11, which prefers '\0' and L'\0', for and C23, which introduced the
new u8'\0', u'\0' (UTF-16), and U'\0' (UTF-32).
c16rtomb, c32rtomb, mbrtoc8: fix page numbers in comments
mbrtoc8(3), mbrtoc16(3), mbrtoc32(3): Say 0 for zero code unit.
Rather than deal with differences between C11 and C23 in notation,
'\0' vs L'\0' vs u8'\0' vs u'\0' vs U'\0'.
uchar.h: Include <sys/featuretest.h> before testing _*_SOURCE.
PR lib/58752: various header files test _*_SOURCE macros but don't
include sys/featuretest.h
PR lib/52374: <uchar.h> missing

uchar.h: Need <sys/cdefs.h> for __restrict.
PR lib/52374: <uchar.h> missing

uchar.h: Simplify __cpp_char8_t and __cplusplus conditionals.
No functional change intended.
PR lib/52374: <uchar.h> missing

tests/lib/libc/locale/t_uchar: Test for char8_t, mbrtoc8, c8rtomb.
PR lib/58752: various header files test _*_SOURCE macros but don't
include sys/featuretest.h
PR lib/52374: <uchar.h> missing

tests/t_uchar: fix copy-and-paste typo
 1.65.6.1  11-Oct-2024  martin Pull up following revision(s) (requested by riastradh in ticket #947):

lib/libc/locale/mbsrtowcs.3: revision 1.14
lib/libc/locale/mbsrtowcs.3: revision 1.15
lib/libc/locale/wcsrtombs.3: revision 1.14
lib/libc/locale/wcsrtombs.3: revision 1.15
lib/libc/locale/wcsrtombs.3: revision 1.16
distrib/sets/lists/comp/mi: revision 1.2473
lib/libc/locale/Makefile.inc: revision 1.68
lib/libc/locale/mbsrtowcs.3: revision 1.13

mbsnrtowcs(3), wcsnrtombs(3): Add man pages for these mouthfuls.

Based on patch drafted by abhinav@, with a little additional tidying
by me.

PR standards/52343: wcsnrtombs missing man-page

mbsnrtowcs(3), wcsnrtombs(3): brush ip markup

SYNOPSIS fixes, uppercase NUL and NULL
 1.68.2.1  01-Oct-2025  martin Pull up following revision(s) (requested by riastradh in ticket #45):

tests/lib/libc/gen/t_ctype.c: revision 1.12
lib/libc/gen/ctype_.c: revision 1.24
lib/libc/locale/rune.c: revision 1.50
tests/lib/libc/gen/Makefile: revision 1.61
lib/libc/gen/tolower_.c: revision 1.18
lib/libc/gen/isctype.c: revision 1.29
distrib/sets/lists/tests/mi: revision 1.1394
lib/libc/gen/toupper_.c: revision 1.18
lib/libc/gen/ctype_guard.h: revision 1.8
lib/libc/locale/Makefile.inc: revision 1.69
lib/libc/gen/ctype.3: revision 1.32
lib/libc/gen/ctype.3: revision 1.33
distrib/sets/lists/debug/mi: revision 1.486
tests/lib/libc/gen/h_ctype_abuse.c: revision 1.1
tests/lib/libc/gen/h_ctype_abuse.c: revision 1.2

ctype(3): New environment variable LIBC_ALLOWCTYPEABUSE.

If set, this does not force the ctype(3) functions to crash when
passed invalid inputs -- instead, they will return nonsense results,
and possibly print warnings to stderr, as is their right in
implementing undefined behaviour.

The nature of the nonsense results is unspecified. Currently, is*()
will always return true (even if that leads to mutually contradictory
conclusions, like isalpha and isdigit, or isgraph and isblank), and
tolower/toupper() will always return EOF. But perhaps in the future
the results may be randomized.

This way, if an application like firefox crashes on ctype abuse, you
can opt to accept the consequences of nonsense results instead by
running `env LIBC_ALLOWCTYPEABUSE= firefox' until the application is
fixed.

PR lib/58208: ctype(3) provides poor runtime feedback of abuse
ctype(3): Document LIBC_ALLOWCTYPEABUSE.

If this is pulled up to netbsd-11, we should tweak the text to make
it apply to 11 too.
PR lib/58208: ctype(3) provides poor runtime feedback of abuse

ctype(3): Fix build of tests on machines with unsigned char.
Could maybe phrase this better but this'll do for now.

PR lib/58208: ctype(3) provides poor runtime feedback of abuse

RSS XML Feed