Undo previous revert, and fix include ordering in prop_extern.c and
prop_intern.c that broke tools build on some hosts.

Revert previous while I investigate the build failure.

Separate the {extern,intern}alize support routines into their own files,
and reduce the verbosity of the names.

NFCI.

branches: 1.10.16; 1.10.36;
The local copy of rb_tree code has been unused for quite some time. So
we can just remove it, and adjust callers to use the "real" rbtree
function names.

Addresses PR lib/44090

Bit-un-rot the proplib portability layer.

branches: 1.8.2;
Change tree op members/typedefs to rbto_compare_* from rb_compare_*

Add prop_array_util functions to proplib. This code is copied/changed
prop_dictionary_util.

From manual page

The prop_array_util family of functions are provided to make getting and
setting values in arrays more convenient in some applications.

OK by mjf@ and freza@.

branches: 1.6.8; 1.6.10;
libprop is currently using a recursive parser. While this is fine for
userland, deeply nested arrays and dictionaries can easily overflow
the kernel stack and thereby force a panic.

Fix the internalizer and prop_object_release to use a separate call
stack and alter the dictionary and array handling to not recurse on
the C stack. The default stack has an inline depth of 16 elements,
which should keep the overhead reasonable.

This issue was found by Pavel Cahyna and Jachym Holecek.

Additionally add a limit for prop_object_copyin_ioctl to prevent user
programs from temporary allocating unbound amount of kernel memory.
Allow malloc to fail so that tight loops of userland processes can't
force panics by exhausting the kernel map.

Tested with the sample exploit of Jachym, his test suite and reviewed
by himself (initial patch), Christos Zoulas and Jason Thorpe.

branches: 1.5.2; 1.5.4; 1.5.12;
- Add prop_dictionary_all_keys(). Takes a snapshot of a dictionary's
keys and returns them in an array. This effectively allows a caller
to mutate a dictionary while iterating over it (really, you iterate
over the array of keys instead of the dictionary itself).
- Add a slew of utility functions that make it more convenient (in some
circumstances, anyway) to get/set values in a dictionary.

Adapt Matt Thomas's red-black tree implementation for use in proplib. This
has a significant code size savings over <sys/tree.h>.

Also change prop_number_t to store all number objects in an r-b tree,
only ever allocating one object for any given number (we can do this
because numbers are immutable). This results in significant run-time
memory savings.

Several changes to proplib:
- Arrays can now be externalized and internalized in the same way
dictionaries can.
- Add new "externalize to file" and "internalize from file" functions
to make reading a property list from a file and writing a property
list to a file more convenient.
- Many assertions in the object implementations are gone. Instead,
calling an accessor for one object type with a different object type
as an argument will return a suitable "invalid" value.
- prop_object_type() now returns a new PROP_TYPE_UNKNOWN value if called
with a NULL object.
- Externalized property lists now contain a reference to the Apple XML
plist DTD.
- Add a new prop_ingest(3) facility, which provides a convenient way to
translate a dictionary into an arbitrary binary representation.

branches: 1.2.2;
Add kernel and user space parts of a protocol for sending property lists
to / from the kernel using ioctls.

Initial commit of proplib, a library for manipulating property lists and
converting to/from an XML external representation (based on Apple XML
property lists). Works in the kernel and user space.

Add support for serializing to, and de-serializing from, JSON, as
specified in RFC 8259. Format is automatically detected for input,
and can be specified explicitly for output. The default format remains
XML.

As part of this change, generic prop_object_{internalize,externalize}()
functions are provided, and the "_with_format" flavor allows the output
format to be specified. Previous array- and dictionary-specific routines
remain for backwards compatibility.

branches: 1.15.12;
Improvements to the problib(3) API:

==> Provide a much more complete set of setters and getters for different
value types in the prop_array_util(3) and prop_dictionary_util(3)
functions.

==> Overhaul the prop_data(3), prop_number(3), and prop_string(3) APIs
to be easier to use and less awkwardly named, Deprecate the old
awkward names, and produce link-time warnings when they are referenced.

==> Deprecate mutable prop_data(3) and prop_string(3) objects. The old
APIs that support them still exist, but will now produce link-time
warnings when used.

==> When the new prop_string(3) API is used, strings are internally
de-duplicated as a memory footprint optimization.

==> Provide a rich set of bounds-checked gettter functions in and a
corresponding set of convenience setters in the prop_number(3) API.

==> Add a new prop_bool_value(3) function that is equivalent to
prop_bool_true(3), but aligned with the new "value" routines in
prop_data(3), prop_string(3), and prop_number(3).

branches: 1.14.12;
Use .Fn to refer a function name instead of Nm.
Also, use Xr to refer problib(3) in the HISTORY section instead of Nm.

While parsing the man pages, any .Nm occurrence gets replaced by the
value specified in the NAME section. Referencing individual function
names with .Nm was causing makemandb(8) to replicate the complete
NAME section for every such occurrence. This was leading to an
ugly looking snippet in apropos(1)'s output when searched for
any of these man pages.

branches: 1.13.26; 1.13.30;
Fix an implementation inconsistency with the prop_*_send_syscall() and
prop_*_recv_syscall() functions from proplib(3). They now share the
same logic as the one from prop_*_send_ioctl() functions:
- returns an int
- 0 indicates "no error", otherwise returns the error number (and
set errno)

Many consumers of the prop_*_{ioctl, syscall} expect errno to be set
on error and use err() to display the error message. As such, ensures that
errno gets set before returning from these functions.

prop_*_send_syscall() functions returned a boolean, and now return an int.
Fix all call sites to use the new paradigm (only quota2 is affected in src).

As the prop_*_{send,recv}_syscall() API appeared in -current and is only
used by the recent quota2 code, I am not bumping the lib. The API change
only affects the prop_*_send_syscall() function (recv_syscall()s were
already used correctly), so ensure you are not mixing "old" -current
quota binaries with a new proplib(3) (or the other way around). This
change will be announced via a HEADS-UP and UPDATING.

Does not affect the kernel part of proplib.

Document the correct API in prop_array(3) and prop_dictionary(3).

Thanks to Francois Tigeot for noticing the API inconsistency and
reporting it on tech-kern@.

ok bouyer@.

Fix section for umask xrefs.

branches: 1.11.4;
Fix some ordering inconsistencies, noted by Silas Silva in PR 42415.

Implement prop_array_externalize_to_pref(), prop_array_copyin(),
prop_dictionary_externalize_to_pref() and prop_dictionary_copyin()
as discussed in the thread:
http://mail-index.netbsd.org/tech-kern/2009/07/29/msg005594.html

Bump libprop shlib minor.

Improve markup.

branches: 1.8.8;
Convert TNF licenses to new 2 clause variant

branches: 1.7.4;
PR/37838: Stathis Kamperis: proplib documentation enhancements

Fix typo

branches: 1.5.2;
boolean_t -> bool
TRUE -> true
FALSE -> false

branches: 1.4.4; 1.4.16;
Several changes to proplib:
- Arrays can now be externalized and internalized in the same way
dictionaries can.
- Add new "externalize to file" and "internalize from file" functions
to make reading a property list from a file and writing a property
list to a file more convenient.
- Many assertions in the object implementations are gone. Instead,
calling an accessor for one object type with a different object type
as an argument will return a suitable "invalid" value.
- prop_object_type() now returns a new PROP_TYPE_UNKNOWN value if called
with a NULL object.
- Externalized property lists now contain a reference to the Apple XML
plist DTD.
- Add a new prop_ingest(3) facility, which provides a convenient way to
translate a dictionary into an arbitrary binary representation.

branches: 1.3.2;
Drop trailing whitespace.

- Add prop_array_equals(), prop_dictionary_equals(), and
prop_dictionary_keysym_equals(), and prop_object_equals() functions.
- Use realloc() where it makes sense. There will be more changes in this
area.
- Add a _prop_object_type structure that is used internally to keep
information about the object types. Decreases the footprint of the
objects slightly by replacing several pointers with just one.

Initial commit of proplib, a library for manipulating property lists and
converting to/from an XML external representation (based on Apple XML
property lists). Works in the kernel and user space.

Undo previous revert, and fix include ordering in prop_extern.c and
prop_intern.c that broke tools build on some hosts.

Revert previous while I investigate the build failure.

Separate the {extern,intern}alize support routines into their own files,
and reduce the verbosity of the names.

NFCI.

Hide the private symbols.

Add support for serializing to, and de-serializing from, JSON, as
specified in RFC 8259. Format is automatically detected for input,
and can be specified explicitly for output. The default format remains
XML.

As part of this change, generic prop_object_{internalize,externalize}()
functions are provided, and the "_with_format" flavor allows the output
format to be specified. Previous array- and dictionary-specific routines
remain for backwards compatibility.

branches: 1.22.6;
fix various typos in documentation, comments and sysctl device description.
mainly aion -> ation and inlude -> include.

branches: 1.21.34;
Bit-un-rot the proplib portability layer.

branches: 1.20.2;
typo in comment and white-space fixes mostly from Stathis Kamperis

Fix an issue introduced with the prop_stack change: It's not a good idea to
return a 3-value enum from a function declared to return bool. This broke
the recurse case for prop_object_equals(). Instead, declare the object type
equals routine to return a _prop_object_equals_rv_t.

Give the same treatment to the object type free routines: declare them to
return a _prop_object_free_rv_t, and consistently check those return values
againt the enum type.

Tidy up some whitespace while we're here.

fix recursive read-lock bugs found by lockdebug and reported
by Geoff C. Wing and others. ok'ed by Jason Thorpe.
an earlier version was tested by Sverre Froyen.
PR/38732 from oster@.

revert the recent "use trylock to hide locking errors" changes
and their associated changes, in particular, the following versions.
prop_array.c 1.13-1.16
prop_dictionary.c 1.25-1.28
prop_object_impl.h 1.21-1.25
ok'ed by core@.

branches: 1.16.2;
Change the argument of _PROP_RWLOCK_TRYRDLOCK() to the lock member and not
its address to be consistent with all the other macros.

Replace _prop_rwlock_tryrdlock() by a macro _PROP_RWLOCK_TRYRDLOCK().
This avoids build failures in stand alone binaries (e.g. bootloaders)
where there is no "pa_rwlock" member in the "_prop_array" structure.

prop_object_iterator_next() and prop_object_iterator_reset() were
acquiring the rwlock recursively in some paths.

Introduce _prop_rwlock_tryrdlock() and use it in these functions, so
that the rwlock is *always* acquired once, while here add some
_PROP_RWLOCK_OWNED asserts to verify.

This definitely fixes the "locking against myself" panics.

Do not acquire the rwlock recursively in prop_{array,dictionary}_iterator_reset()
when prop_{array,dictionary}_copyout_ioctl() is called.

Introduce _PROP_RWLOCK_OWNED() which is a KASSERT(rw_lock_held(lock))
and use it in those two functions, also acquire the rwlock in other
places where it is required now.

This fixes a LOCKDEBUG panic "locking against myself", as reported by
Geoff C. Wing in current-users@.

Remove clause 3 and 4 from TNF licenses

branches: 1.11.6;
Avoid using unbound amount of stack frames in prop_object_equal
by using a dynamic stack as well. Reorder arguments for the internalizer
as the iteration is always present and should go before possibly
NULL arguments.

Reviewed by mjf@ and adrianp@

branches: 1.10.2;
libprop is currently using a recursive parser. While this is fine for
userland, deeply nested arrays and dictionaries can easily overflow
the kernel stack and thereby force a panic.

Fix the internalizer and prop_object_release to use a separate call
stack and alter the dictionary and array handling to not recurse on
the C stack. The default stack has an inline depth of 16 elements,
which should keep the overhead reasonable.

This issue was found by Pavel Cahyna and Jachym Holecek.

Additionally add a limit for prop_object_copyin_ioctl to prevent user
programs from temporary allocating unbound amount of kernel memory.
Allow malloc to fail so that tight loops of userland processes can't
force panics by exhausting the kernel map.

Tested with the sample exploit of Jachym, his test suite and reviewed
by himself (initial patch), Christos Zoulas and Jason Thorpe.

boolean_t -> bool
TRUE -> true
FALSE -> false

prop_array_internalize and prop_dictionary_internalize are basically the
same code. Refactor it into _prop_generic_internalize, which gets passed
the second-level tag.

branches: 1.7.2; 1.7.4; 1.7.12;
- Add a reader-writer lock to array and dictionary objects that ensures that
the data structure is internally consistent in the face of multiple threads
accessing it concurrently. This is not designed to provide application-
level semantic consistency; applications are responsible for that locking
protocol should it be necessary.
- Rename _PROP_MUTEX_DECL() to _PROP_MUTEX_DECL_STATIC().

Also guard object accessors against NULL (like we do foreign object types)
so that apps can use this construct safely:

obj = prop_dictionary_get(dict, "value");
if (! prop_number_equals_integer(obj, 5)) {
...
}

Suggested by Iain Hibbert.

Add a workaround for a GCC -Wuninitialized mis-warning (seen for mips).

Several changes to proplib:
- Arrays can now be externalized and internalized in the same way
dictionaries can.
- Add new "externalize to file" and "internalize from file" functions
to make reading a property list from a file and writing a property
list to a file more convenient.
- Many assertions in the object implementations are gone. Instead,
calling an accessor for one object type with a different object type
as an argument will return a suitable "invalid" value.
- prop_object_type() now returns a new PROP_TYPE_UNKNOWN value if called
with a NULL object.
- Externalized property lists now contain a reference to the Apple XML
plist DTD.
- Add a new prop_ingest(3) facility, which provides a convenient way to
translate a dictionary into an arbitrary binary representation.

branches: 1.3.2;
Fix the way that prop_array_ensure_capacity() and _prop_array_expand()
interact.

- Add prop_array_equals(), prop_dictionary_equals(), and
prop_dictionary_keysym_equals(), and prop_object_equals() functions.
- Use realloc() where it makes sense. There will be more changes in this
area.
- Add a _prop_object_type structure that is used internally to keep
information about the object types. Decreases the footprint of the
objects slightly by replacing several pointers with just one.

Initial commit of proplib, a library for manipulating property lists and
converting to/from an XML external representation (based on Apple XML
property lists). Works in the kernel and user space.

Fix various typos in comments, log messages and documentation.

fix various typos in documentation, comments and sysctl device description.
mainly aion -> ation and inlude -> include.

Improvements to the problib(3) API:

==> Provide a much more complete set of setters and getters for different
value types in the prop_array_util(3) and prop_dictionary_util(3)
functions.

==> Overhaul the prop_data(3), prop_number(3), and prop_string(3) APIs
to be easier to use and less awkwardly named, Deprecate the old
awkward names, and produce link-time warnings when they are referenced.

==> Deprecate mutable prop_data(3) and prop_string(3) objects. The old
APIs that support them still exist, but will now produce link-time
warnings when used.

==> When the new prop_string(3) API is used, strings are internally
de-duplicated as a memory footprint optimization.

==> Provide a rich set of bounds-checked gettter functions in and a
corresponding set of convenience setters in the prop_number(3) API.

==> Add a new prop_bool_value(3) function that is equivalent to
prop_bool_true(3), but aligned with the new "value" routines in
prop_data(3), prop_string(3), and prop_number(3).

branches: 1.10.12;
Use .Fn to refer a function name instead of Nm.
Also, use Xr to refer problib(3) in the HISTORY section instead of Nm.

While parsing the man pages, any .Nm occurrence gets replaced by the
value specified in the NAME section. Referencing individual function
names with .Nm was causing makemandb(8) to replicate the complete
NAME section for every such occurrence. This was leading to an
ugly looking snippet in apropos(1)'s output when searched for
any of these man pages.

branches: 1.9.2; 1.9.4;
Punctuation nits.
(adding space makes sure they are not marked up)

Add prop_array_add_cstring{,_no_copy} utility functions as suggested in
PR lib/46723 using the patch provided. (Header file updates are mine.)

Add missing Nd.

Fix a typo and a punctuation nit.

Add a new libquota library, which contains some blocks to build and/or
parse quota plists; as well as a getfsquota() function to retrieve quotas
for a single id from a single filesystem (whatever filesystem this is:
a local quota-enabled fs or NFS). This is build on functions getufsquota()
(for local filesystems with UFS-like quotas) and getnfsquota();
which are also available to userland programs.
move functions from quota2_subr.c to libquota or libprop as appropriate,
and ajust in-tree quota tools.
move some declarations from kernel headers to either sys/quota.h or
quota/quota.h as appropriate. ufs/ufs/quota.h still installed because
it's needed by other installed ufs headers.
ufs/ufs/quota1.h still installed as a quick&dirty way to get a code
using the old quotactl() to compile (just include ufs/ufs/quota1.h instead of
ufs/ufs/quota.h - old code won't compile without this change and this is
on purpose).
Discussed on tech-kern@ and tech-net@ (long thread, but not much about
libquota itself ...)

Remove trailing whitespace.

Add prop_array_add_int* and prop_array_add_uint* functions. These functions
can be used to append specified type to the end of prop_array_t.

Ok'ed by @joerg.

branches: 1.2.2; 1.2.4;
Sort SEE ALSO.

Add prop_array_util functions to proplib. This code is copied/changed
prop_dictionary_util.

From manual page

The prop_array_util family of functions are provided to make getting and
setting values in arrays more convenient in some applications.

OK by mjf@ and freza@.

Hide the private symbols.

branches: 1.9.8;
proplib: Nix trailing whitespace.

add/fix linted comments

Fix incorrect type (found by lint), and add linted comments for the
long long -> long (uintmax_t on LP64)

Improvements to the problib(3) API:

==> Provide a much more complete set of setters and getters for different
value types in the prop_array_util(3) and prop_dictionary_util(3)
functions.

==> Overhaul the prop_data(3), prop_number(3), and prop_string(3) APIs
to be easier to use and less awkwardly named, Deprecate the old
awkward names, and produce link-time warnings when they are referenced.

==> Deprecate mutable prop_data(3) and prop_string(3) objects. The old
APIs that support them still exist, but will now produce link-time
warnings when used.

==> When the new prop_string(3) API is used, strings are internally
de-duplicated as a memory footprint optimization.

==> Provide a rich set of bounds-checked gettter functions in and a
corresponding set of convenience setters in the prop_number(3) API.

==> Add a new prop_bool_value(3) function that is equivalent to
prop_bool_true(3), but aligned with the new "value" routines in
prop_data(3), prop_string(3), and prop_number(3).

branches: 1.5.16;
Add prop_array_add_cstring{,_no_copy} utility functions as suggested in
PR lib/46723 using the patch provided. (Header file updates are mine.)

Bit-un-rot the proplib portability layer.

branches: 1.3.4;
Add a new libquota library, which contains some blocks to build and/or
parse quota plists; as well as a getfsquota() function to retrieve quotas
for a single id from a single filesystem (whatever filesystem this is:
a local quota-enabled fs or NFS). This is build on functions getufsquota()
(for local filesystems with UFS-like quotas) and getnfsquota();
which are also available to userland programs.
move functions from quota2_subr.c to libquota or libprop as appropriate,
and ajust in-tree quota tools.
move some declarations from kernel headers to either sys/quota.h or
quota/quota.h as appropriate. ufs/ufs/quota.h still installed because
it's needed by other installed ufs headers.
ufs/ufs/quota1.h still installed as a quick&dirty way to get a code
using the old quotactl() to compile (just include ufs/ufs/quota1.h instead of
ufs/ufs/quota.h - old code won't compile without this change and this is
on purpose).
Discussed on tech-kern@ and tech-net@ (long thread, but not much about
libquota itself ...)

Add prop_array_add_int* and prop_array_add_uint* functions. These functions
can be used to append specified type to the end of prop_array_t.

Ok'ed by @joerg.

branches: 1.1.2; 1.1.4;
Add prop_array_util functions to proplib. This code is copied/changed
prop_dictionary_util.

From manual page

The prop_array_util family of functions are provided to make getting and
setting values in arrays more convenient in some applications.

OK by mjf@ and freza@.

Improvements to the problib(3) API:

==> Provide a much more complete set of setters and getters for different
value types in the prop_array_util(3) and prop_dictionary_util(3)
functions.

==> Overhaul the prop_data(3), prop_number(3), and prop_string(3) APIs
to be easier to use and less awkwardly named, Deprecate the old
awkward names, and produce link-time warnings when they are referenced.

==> Deprecate mutable prop_data(3) and prop_string(3) objects. The old
APIs that support them still exist, but will now produce link-time
warnings when used.

==> When the new prop_string(3) API is used, strings are internally
de-duplicated as a memory footprint optimization.

==> Provide a rich set of bounds-checked gettter functions in and a
corresponding set of convenience setters in the prop_number(3) API.

==> Add a new prop_bool_value(3) function that is equivalent to
prop_bool_true(3), but aligned with the new "value" routines in
prop_data(3), prop_string(3), and prop_number(3).

branches: 1.7.12;
Use .Fn to refer a function name instead of Nm.
Also, use Xr to refer problib(3) in the HISTORY section instead of Nm.

While parsing the man pages, any .Nm occurrence gets replaced by the
value specified in the NAME section. Referencing individual function
names with .Nm was causing makemandb(8) to replicate the complete
NAME section for every such occurrence. This was leading to an
ugly looking snippet in apropos(1)'s output when searched for
any of these man pages.

branches: 1.6.44; 1.6.48;
Fix a typo.

branches: 1.5.2;
Convert TNF licenses to new 2 clause variant

branches: 1.4.4;
PR/37838: Stathis Kamperis: proplib documentation enhancements

branches: 1.3.2;
boolean_t -> bool
TRUE -> true
FALSE -> false

branches: 1.2.6; 1.2.18;
Drop trailing whitespace.

Initial commit of proplib, a library for manipulating property lists and
converting to/from an XML external representation (based on Apple XML
property lists). Works in the kernel and user space.

Undo previous revert, and fix include ordering in prop_extern.c and
prop_intern.c that broke tools build on some hosts.

Revert previous while I investigate the build failure.

Separate the {extern,intern}alize support routines into their own files,
and reduce the verbosity of the names.

NFCI.

Hide the private symbols.

Add support for serializing to, and de-serializing from, JSON, as
specified in RFC 8259. Format is automatically detected for input,
and can be specified explicitly for output. The default format remains
XML.

As part of this change, generic prop_object_{internalize,externalize}()
functions are provided, and the "_with_format" flavor allows the output
format to be specified. Previous array- and dictionary-specific routines
remain for backwards compatibility.

branches: 1.19.12;
Improvements to the problib(3) API:

==> Provide a much more complete set of setters and getters for different
value types in the prop_array_util(3) and prop_dictionary_util(3)
functions.

==> Overhaul the prop_data(3), prop_number(3), and prop_string(3) APIs
to be easier to use and less awkwardly named, Deprecate the old
awkward names, and produce link-time warnings when they are referenced.

==> Deprecate mutable prop_data(3) and prop_string(3) objects. The old
APIs that support them still exist, but will now produce link-time
warnings when used.

==> When the new prop_string(3) API is used, strings are internally
de-duplicated as a memory footprint optimization.

==> Provide a rich set of bounds-checked gettter functions in and a
corresponding set of convenience setters in the prop_number(3) API.

==> Add a new prop_bool_value(3) function that is equivalent to
prop_bool_true(3), but aligned with the new "value" routines in
prop_data(3), prop_string(3), and prop_number(3).

Fix building libprop as a host tool library on platforms that don't have
the Matt Thomas rbtree:

- Include rb.c in libnbcompat, and provide a nbcompat sys/rbtree.h
header.
- Make sure libprop's source file include prop_object_impl.h before
anything else, and pull in nbtool_config.h from there.

Tested by simulating such a host system by renaming the host's
<sys/rbtree.h> out of the way (which reproduced the build failure)
and verifying that the host-tool installboot contained the rb_*
functions in its own .text segment.

branches: 1.17.50;
Get rid of locks with static initializers using once and atomic ops.
This makes proplib simplelock-free.

Fix an issue introduced with the prop_stack change: It's not a good idea to
return a 3-value enum from a function declared to return bool. This broke
the recurse case for prop_object_equals(). Instead, declare the object type
equals routine to return a _prop_object_equals_rv_t.

Give the same treatment to the object type free routines: declare them to
return a _prop_object_free_rv_t, and consistently check those return values
againt the enum type.

Tidy up some whitespace while we're here.

branches: 1.15.2;
Remove clause 3 and 4 from TNF licenses

branches: 1.14.4;
Include sys/simplelock.h in prop_object_impl.h.

sys/simplelock.h is required now.

Avoid using unbound amount of stack frames in prop_object_equal
by using a dynamic stack as well. Reorder arguments for the internalizer
as the iteration is always present and should go before possibly
NULL arguments.

Reviewed by mjf@ and adrianp@

branches: 1.11.2;
libprop is currently using a recursive parser. While this is fine for
userland, deeply nested arrays and dictionaries can easily overflow
the kernel stack and thereby force a panic.

Fix the internalizer and prop_object_release to use a separate call
stack and alter the dictionary and array handling to not recurse on
the C stack. The default stack has an inline depth of 16 elements,
which should keep the overhead reasonable.

This issue was found by Pavel Cahyna and Jachym Holecek.

Additionally add a limit for prop_object_copyin_ioctl to prevent user
programs from temporary allocating unbound amount of kernel memory.
Allow malloc to fail so that tight loops of userland processes can't
force panics by exhausting the kernel map.

Tested with the sample exploit of Jachym, his test suite and reviewed
by himself (initial patch), Christos Zoulas and Jason Thorpe.

boolean_t -> bool
TRUE -> true
FALSE -> false

branches: 1.9.2; 1.9.4; 1.9.12;
Add a _PROP_ARG_UNUSED that expands to __unused on NetBSD and to nothing
everywhere else. Use it where Christos decided to use __unused in this
code.

de-lint.

_prop_bool_free(): Consume all of the arguments.

- Add a reader-writer lock to array and dictionary objects that ensures that
the data structure is internally consistent in the face of multiple threads
accessing it concurrently. This is not designed to provide application-
level semantic consistency; applications are responsible for that locking
protocol should it be necessary.
- Rename _PROP_MUTEX_DECL() to _PROP_MUTEX_DECL_STATIC().

Also guard object accessors against NULL (like we do foreign object types)
so that apps can use this construct safely:

obj = prop_dictionary_get(dict, "value");
if (! prop_number_equals_integer(obj, 5)) {
...
}

Suggested by Iain Hibbert.

Several changes to proplib:
- Arrays can now be externalized and internalized in the same way
dictionaries can.
- Add new "externalize to file" and "internalize from file" functions
to make reading a property list from a file and writing a property
list to a file more convenient.
- Many assertions in the object implementations are gone. Instead,
calling an accessor for one object type with a different object type
as an argument will return a suitable "invalid" value.
- prop_object_type() now returns a new PROP_TYPE_UNKNOWN value if called
with a NULL object.
- Externalized property lists now contain a reference to the Apple XML
plist DTD.
- Add a new prop_ingest(3) facility, which provides a convenient way to
translate a dictionary into an arbitrary binary representation.

branches: 1.3.2;
Reduce the memory footprint for prop_bool_t objects significantly, taking
adavantage of the immutability of these objects:

Statically allocate a TRUE object and a FALSE object, and simply return
references to those objects for create and copy operations.

- Add prop_array_equals(), prop_dictionary_equals(), and
prop_dictionary_keysym_equals(), and prop_object_equals() functions.
- Use realloc() where it makes sense. There will be more changes in this
area.
- Add a _prop_object_type structure that is used internally to keep
information about the object types. Decreases the footprint of the
objects slightly by replacing several pointers with just one.

Initial commit of proplib, a library for manipulating property lists and
converting to/from an XML external representation (based on Apple XML
property lists). Works in the kernel and user space.

Add support for serializing to, and de-serializing from, JSON, as
specified in RFC 8259. Format is automatically detected for input,
and can be specified explicitly for output. The default format remains
XML.

As part of this change, generic prop_object_{internalize,externalize}()
functions are provided, and the "_with_format" flavor allows the output
format to be specified. Previous array- and dictionary-specific routines
remain for backwards compatibility.

branches: 1.15.12; 1.15.32;
Fix commas in NAME section.

Add the *_size() variants to the .Nm list so it will be found by
apropos(1).

While here, reorder the function lists to improve readability.

Use just the right amount of commas.

Update for the new *_size() functions recently added. Mention the
implicit size limit (128KB) for the functions which do not take an
explicit limit argument.

Sort errors.

Document E2BIG error when copying a large object.

branches: 1.9.26; 1.9.30; 1.9.34;
Add commas in enumerations.

- Implement prop_{array,dictionary}_copyout(), based on
prop_{array,dictionary}_copyout_ioctl().
- Implement prop_{array,dictionary}_recv_syscall(), which
takes as parameter the pref we got from kernel and internalize it,
and unmaps the buffer prop_{array,dictionary}_copyout() mapped for us.
- add a prop_{array,dictionary}_send_syscall() for symetry, which is
an alias to prop_{array,dictionary}_externalize_to_pref()

Discussed on tech-kern@ and tech-userlevel@

Minor wording fix/clarification (sending -> copying) from Silas Silva
in PR 42414.

Sort sections. New sentence, new line.

Implement prop_array_externalize_to_pref(), prop_array_copyin(),
prop_dictionary_externalize_to_pref() and prop_dictionary_copyin()
as discussed in the thread:
http://mail-index.netbsd.org/tech-kern/2009/07/29/msg005594.html

Bump libprop shlib minor.

Convert TNF licenses to new 2 clause variant

branches: 1.3.4;
PR/37838: Stathis Kamperis: proplib documentation enhancements

branches: 1.2.4;
correct function names.

branches: 1.1.2;
Add prop_array_t support to prop_kern.

prop_data.3: fix typo

Improvements to the problib(3) API:

==> Provide a much more complete set of setters and getters for different
value types in the prop_array_util(3) and prop_dictionary_util(3)
functions.

==> Overhaul the prop_data(3), prop_number(3), and prop_string(3) APIs
to be easier to use and less awkwardly named, Deprecate the old
awkward names, and produce link-time warnings when they are referenced.

==> Deprecate mutable prop_data(3) and prop_string(3) objects. The old
APIs that support them still exist, but will now produce link-time
warnings when used.

==> When the new prop_string(3) API is used, strings are internally
de-duplicated as a memory footprint optimization.

==> Provide a rich set of bounds-checked gettter functions in and a
corresponding set of convenience setters in the prop_number(3) API.

==> Add a new prop_bool_value(3) function that is equivalent to
prop_bool_true(3), but aligned with the new "value" routines in
prop_data(3), prop_string(3), and prop_number(3).

branches: 1.8.12;
Use .Fn to refer a function name instead of Nm.
Also, use Xr to refer problib(3) in the HISTORY section instead of Nm.

While parsing the man pages, any .Nm occurrence gets replaced by the
value specified in the NAME section. Referencing individual function
names with .Nm was causing makemandb(8) to replicate the complete
NAME section for every such occurrence. This was leading to an
ugly looking snippet in apropos(1)'s output when searched for
any of these man pages.

branches: 1.7.30; 1.7.34;
Fix some ordering inconsistencies, noted by Silas Silva in PR 42415.

Convert TNF licenses to new 2 clause variant

branches: 1.5.4;
PR/37838: Stathis Kamperis: proplib documentation enhancements

branches: 1.4.2;
boolean_t -> bool
TRUE -> true
FALSE -> false

Add prop_data_size() to SYNOPSIS.

branches: 1.2.6; 1.2.10;
Drop trailing whitespace.

Initial commit of proplib, a library for manipulating property lists and
converting to/from an XML external representation (based on Apple XML
property lists). Works in the kernel and user space.

libprop: eliminate common subexpression

Undo previous revert, and fix include ordering in prop_extern.c and
prop_intern.c that broke tools build on some hosts.

Revert previous while I investigate the build failure.

Separate the {extern,intern}alize support routines into their own files,
and reduce the verbosity of the names.

NFCI.

Hide the private symbols.

Add support for serializing to, and de-serializing from, JSON, as
specified in RFC 8259. Format is automatically detected for input,
and can be specified explicitly for output. The default format remains
XML.

As part of this change, generic prop_object_{internalize,externalize}()
functions are provided, and the "_with_format" flavor allows the output
format to be specified. Previous array- and dictionary-specific routines
remain for backwards compatibility.

branches: 1.18.8;
proplib: Nix trailing whitespace.

Fix a paste-o that caused prop_data_create_copy() to be intolerant
of creating empty data objects. Fixes t_ifconfig::ifconfig_description
unit test.

Improvements to the problib(3) API:

==> Provide a much more complete set of setters and getters for different
value types in the prop_array_util(3) and prop_dictionary_util(3)
functions.

==> Overhaul the prop_data(3), prop_number(3), and prop_string(3) APIs
to be easier to use and less awkwardly named, Deprecate the old
awkward names, and produce link-time warnings when they are referenced.

==> Deprecate mutable prop_data(3) and prop_string(3) objects. The old
APIs that support them still exist, but will now produce link-time
warnings when used.

==> When the new prop_string(3) API is used, strings are internally
de-duplicated as a memory footprint optimization.

==> Provide a rich set of bounds-checked gettter functions in and a
corresponding set of convenience setters in the prop_number(3) API.

==> Add a new prop_bool_value(3) function that is equivalent to
prop_bool_true(3), but aligned with the new "value" routines in
prop_data(3), prop_string(3), and prop_number(3).

Fix building libprop as a host tool library on platforms that don't have
the Matt Thomas rbtree:

- Include rb.c in libnbcompat, and provide a nbcompat sys/rbtree.h
header.
- Make sure libprop's source file include prop_object_impl.h before
anything else, and pull in nbtool_config.h from there.

Tested by simulating such a host system by renaming the host's
<sys/rbtree.h> out of the way (which reproduced the build failure)
and verifying that the host-tool installboot contained the rb_*
functions in its own .text segment.

branches: 1.14.50;
Properly handle empty data nodes (such as generated by Apple's
plist editor). They are in the form of <data></data>, whereas the
original code path only checked for empty being <data/>.

This causes an assert to be triggered when trying to access the node's
content.

Patch reviewed by thorpej.

branches: 1.13.4;
Fix an issue introduced with the prop_stack change: It's not a good idea to
return a 3-value enum from a function declared to return bool. This broke
the recurse case for prop_object_equals(). Instead, declare the object type
equals routine to return a _prop_object_equals_rv_t.

Give the same treatment to the object type free routines: declare them to
return a _prop_object_free_rv_t, and consistently check those return values
againt the enum type.

Tidy up some whitespace while we're here.

In prop_data_create_data(blob, len), if len == 0, allocate no memory
for blob, and leave pd_mutable set to NULL. In this way, avoid
tripping over the assertion in prop_data_data_nocopy().

branches: 1.11.2;
Remove clause 3 and 4 from TNF licenses

branches: 1.10.4;
For all files under src/common, as well as related man pages under
src/lib/libc, or related headers under src/sys/sys: change u_intNN_t to
uintNN_t.

Avoid using unbound amount of stack frames in prop_object_equal
by using a dynamic stack as well. Reorder arguments for the internalizer
as the iteration is always present and should go before possibly
NULL arguments.

Reviewed by mjf@ and adrianp@

branches: 1.8.2;
libprop is currently using a recursive parser. While this is fine for
userland, deeply nested arrays and dictionaries can easily overflow
the kernel stack and thereby force a panic.

Fix the internalizer and prop_object_release to use a separate call
stack and alter the dictionary and array handling to not recurse on
the C stack. The default stack has an inline depth of 16 elements,
which should keep the overhead reasonable.

This issue was found by Pavel Cahyna and Jachym Holecek.

Additionally add a limit for prop_object_copyin_ioctl to prevent user
programs from temporary allocating unbound amount of kernel memory.
Allow malloc to fail so that tight loops of userland processes can't
force panics by exhausting the kernel map.

Tested with the sample exploit of Jachym, his test suite and reviewed
by himself (initial patch), Christos Zoulas and Jason Thorpe.

boolean_t -> bool
TRUE -> true
FALSE -> false

Fix white space skipping before closing tag when padding characters
are present.

Okayed by thorpej.

branches: 1.5.2; 1.5.4; 1.5.6;
Add casts to apease lint.

Also guard object accessors against NULL (like we do foreign object types)
so that apps can use this construct safely:

obj = prop_dictionary_get(dict, "value");
if (! prop_number_equals_integer(obj, 5)) {
...
}

Suggested by Iain Hibbert.

Several changes to proplib:
- Arrays can now be externalized and internalized in the same way
dictionaries can.
- Add new "externalize to file" and "internalize from file" functions
to make reading a property list from a file and writing a property
list to a file more convenient.
- Many assertions in the object implementations are gone. Instead,
calling an accessor for one object type with a different object type
as an argument will return a suitable "invalid" value.
- prop_object_type() now returns a new PROP_TYPE_UNKNOWN value if called
with a NULL object.
- Externalized property lists now contain a reference to the Apple XML
plist DTD.
- Add a new prop_ingest(3) facility, which provides a convenient way to
translate a dictionary into an arbitrary binary representation.

branches: 1.2.2;
- Add prop_array_equals(), prop_dictionary_equals(), and
prop_dictionary_keysym_equals(), and prop_object_equals() functions.
- Use realloc() where it makes sense. There will be more changes in this
area.
- Add a _prop_object_type structure that is used internally to keep
information about the object types. Decreases the footprint of the
objects slightly by replacing several pointers with just one.

Initial commit of proplib, a library for manipulating property lists and
converting to/from an XML external representation (based on Apple XML
property lists). Works in the kernel and user space.

Add support for serializing to, and de-serializing from, JSON, as
specified in RFC 8259. Format is automatically detected for input,
and can be specified explicitly for output. The default format remains
XML.

As part of this change, generic prop_object_{internalize,externalize}()
functions are provided, and the "_with_format" flavor allows the output
format to be specified. Previous array- and dictionary-specific routines
remain for backwards compatibility.

prop_dictionary.3 & prop_string.3: minor grammar fixes

branches: 1.20.12;
Improvements to the problib(3) API:

==> Provide a much more complete set of setters and getters for different
value types in the prop_array_util(3) and prop_dictionary_util(3)
functions.

==> Overhaul the prop_data(3), prop_number(3), and prop_string(3) APIs
to be easier to use and less awkwardly named, Deprecate the old
awkward names, and produce link-time warnings when they are referenced.

==> Deprecate mutable prop_data(3) and prop_string(3) objects. The old
APIs that support them still exist, but will now produce link-time
warnings when used.

==> When the new prop_string(3) API is used, strings are internally
de-duplicated as a memory footprint optimization.

==> Provide a rich set of bounds-checked gettter functions in and a
corresponding set of convenience setters in the prop_number(3) API.

==> Add a new prop_bool_value(3) function that is equivalent to
prop_bool_true(3), but aligned with the new "value" routines in
prop_data(3), prop_string(3), and prop_number(3).

branches: 1.19.12;
Use .Fn to refer a function name instead of Nm.
Also, use Xr to refer problib(3) in the HISTORY section instead of Nm.

While parsing the man pages, any .Nm occurrence gets replaced by the
value specified in the NAME section. Referencing individual function
names with .Nm was causing makemandb(8) to replicate the complete
NAME section for every such occurrence. This was leading to an
ugly looking snippet in apropos(1)'s output when searched for
any of these man pages.

branches: 1.18.26; 1.18.30;
Fix an implementation inconsistency with the prop_*_send_syscall() and
prop_*_recv_syscall() functions from proplib(3). They now share the
same logic as the one from prop_*_send_ioctl() functions:
- returns an int
- 0 indicates "no error", otherwise returns the error number (and
set errno)

Many consumers of the prop_*_{ioctl, syscall} expect errno to be set
on error and use err() to display the error message. As such, ensures that
errno gets set before returning from these functions.

prop_*_send_syscall() functions returned a boolean, and now return an int.
Fix all call sites to use the new paradigm (only quota2 is affected in src).

As the prop_*_{send,recv}_syscall() API appeared in -current and is only
used by the recent quota2 code, I am not bumping the lib. The API change
only affects the prop_*_send_syscall() function (recv_syscall()s were
already used correctly), so ensure you are not mixing "old" -current
quota binaries with a new proplib(3) (or the other way around). This
change will be announced via a HEADS-UP and UPDATING.

Does not affect the kernel part of proplib.

Document the correct API in prop_array(3) and prop_dictionary(3).

Thanks to Francois Tigeot for noticing the API inconsistency and
reporting it on tech-kern@.

ok bouyer@.

Fix section for umask xrefs.

prop_dictionary_internalize_from_file_returns_a_dictionary_not_an_array

branches: 1.15.4;
Fix prop_dictionary_ensure_capacity signature, bump date.
From Silas Silva in PR 42413.

Implement prop_array_externalize_to_pref(), prop_array_copyin(),
prop_dictionary_externalize_to_pref() and prop_dictionary_copyin()
as discussed in the thread:
http://mail-index.netbsd.org/tech-kern/2009/07/29/msg005594.html

Bump libprop shlib minor.

Xref prop_dictionary_util(3).

Convert TNF licenses to new 2 clause variant

branches: 1.11.4;
PR/37838: Stathis Kamperis: proplib documentation enhancements

Nuke prop_dictionary_capacity, there was no implementation for it.

No objections on tech-kern.

Fix a typo that has been there forever. prop_dictionary_set:
the object will be retained by the dictionary not array.

branches: 1.8.2;
boolean_t -> bool
TRUE -> true
FALSE -> false

Fix the prototyp of prop_dictionary_set_keysym in the DESCRIPTION part

It's "Copy a dictionary" not "Copy an dictionary". preempt wiz and
bump date.

branches: 1.5.2; 1.5.4; 1.5.14;
- Add prop_dictionary_all_keys(). Takes a snapshot of a dictionary's
keys and returns them in an array. This effectively allows a caller
to mutate a dictionary while iterating over it (really, you iterate
over the array of keys instead of the dictionary itself).
- Add a slew of utility functions that make it more convenient (in some
circumstances, anyway) to get/set values in a dictionary.

Several changes to proplib:
- Arrays can now be externalized and internalized in the same way
dictionaries can.
- Add new "externalize to file" and "internalize from file" functions
to make reading a property list from a file and writing a property
list to a file more convenient.
- Many assertions in the object implementations are gone. Instead,
calling an accessor for one object type with a different object type
as an argument will return a suitable "invalid" value.
- prop_object_type() now returns a new PROP_TYPE_UNKNOWN value if called
with a NULL object.
- Externalized property lists now contain a reference to the Apple XML
plist DTD.
- Add a new prop_ingest(3) facility, which provides a convenient way to
translate a dictionary into an arbitrary binary representation.

branches: 1.3.2;
Drop trailing whitespace.

- Add prop_array_equals(), prop_dictionary_equals(), and
prop_dictionary_keysym_equals(), and prop_object_equals() functions.
- Use realloc() where it makes sense. There will be more changes in this
area.
- Add a _prop_object_type structure that is used internally to keep
information about the object types. Decreases the footprint of the
objects slightly by replacing several pointers with just one.

Initial commit of proplib, a library for manipulating property lists and
converting to/from an XML external representation (based on Apple XML
property lists). Works in the kernel and user space.

Undo previous revert, and fix include ordering in prop_extern.c and
prop_intern.c that broke tools build on some hosts.

Revert previous while I investigate the build failure.

Separate the {extern,intern}alize support routines into their own files,
and reduce the verbosity of the names.

NFCI.

Hide the private symbols.

Add support for serializing to, and de-serializing from, JSON, as
specified in RFC 8259. Format is automatically detected for input,
and can be specified explicitly for output. The default format remains
XML.

As part of this change, generic prop_object_{internalize,externalize}()
functions are provided, and the "_with_format" flavor allows the output
format to be specified. Previous array- and dictionary-specific routines
remain for backwards compatibility.

branches: 1.46.6;
Sprinkle braces around _PROP_RWLOCK_UNLOCK() in ``if'' block.
It is expanded into /* nothing */ for _STANDALONE.

proplib: Nix trailing whitespace.

s/refrences/references/

s/exisit/exist/ in comment.

Improvements to the problib(3) API:

==> Provide a much more complete set of setters and getters for different
value types in the prop_array_util(3) and prop_dictionary_util(3)
functions.

==> Overhaul the prop_data(3), prop_number(3), and prop_string(3) APIs
to be easier to use and less awkwardly named, Deprecate the old
awkward names, and produce link-time warnings when they are referenced.

==> Deprecate mutable prop_data(3) and prop_string(3) objects. The old
APIs that support them still exist, but will now produce link-time
warnings when used.

==> When the new prop_string(3) API is used, strings are internally
de-duplicated as a memory footprint optimization.

==> Provide a rich set of bounds-checked gettter functions in and a
corresponding set of convenience setters in the prop_number(3) API.

==> Add a new prop_bool_value(3) function that is equivalent to
prop_bool_true(3), but aligned with the new "value" routines in
prop_data(3), prop_string(3), and prop_number(3).

branches: 1.41.16;
Missed a couple of function-call renames in previous

The local copy of rb_tree code has been unused for quite some time. So
we can just remove it, and adjust callers to use the "real" rbtree
function names.

Addresses PR lib/44090

Make this compilable with gcc 4.8.1 without options DIAGNOSTIC.

branches: 1.38.2;
Bit-un-rot the proplib portability layer.

branches: 1.37.4;
Fix "address of a void*" vs. void* confusion.
Part of fixing PR lib/43964.

Fixes/improvements to RB-tree implementation:
1. Fix inverted node order, so that negative value from comparison operator
would represent lower (left) node, and positive - higher (right) node.
2. Add an argument (i.e. "context"), passed to comparison operators.
3. Change rb_tree_insert_node() to return a node - either inserted one or
already existing one.
4. Amend the interface to manipulate the actual object, instead of the
rb_node (in a similar way as Patricia-tree interface does).
5. Update all RB-tree users accordingly.

XXX: Perhaps rename rb.h to rbtree.h, since cleaning-up..

1-3 address the PR/43488 by Jeremy Huddleston.

Passes RB-tree regression tests.
Reviewed by: matt@, christos@

Check if pd is not NULL before we try to lock rw lock associated with it.
This fixes proplib crash when NULL is passed to prop_dictionary_get as a
dictionary.

branches: 1.34.2;
Get rid of locks with static initializers using once and atomic ops.
This makes proplib simplelock-free.

Fix two race conditions in proplib library. In prop_dictionary and prop_number
there was small window in which was entry left in rb tree with reference
count 0 which lead to rb tree coruption when another thread picked this up
before it was released.
Add 2 new members to the prop_object_t which are used for locking/unlocking
rb tree guard mutex.

Ok by joerg@, core@, thorpej@

branches: 1.32.4;
Fix an issue introduced with the prop_stack change: It's not a good idea to
return a 3-value enum from a function declared to return bool. This broke
the recurse case for prop_object_equals(). Instead, declare the object type
equals routine to return a _prop_object_equals_rv_t.

Give the same treatment to the object type free routines: declare them to
return a _prop_object_free_rv_t, and consistently check those return values
againt the enum type.

Tidy up some whitespace while we're here.

_prop_rb_tree_insert_node() now returns true/false to indicate if the
insertion succeeded. Update existing usage that arranges for insertions
to always succeed to assert that they do.

fix recursive read-lock bugs found by lockdebug and reported
by Geoff C. Wing and others. ok'ed by Jason Thorpe.
an earlier version was tested by Sverre Froyen.
PR/38732 from oster@.

revert the recent "use trylock to hide locking errors" changes
and their associated changes, in particular, the following versions.
prop_array.c 1.13-1.16
prop_dictionary.c 1.25-1.28
prop_object_impl.h 1.21-1.25
ok'ed by core@.

branches: 1.28.2;
Change the argument of _PROP_RWLOCK_TRYRDLOCK() to the lock member and not
its address to be consistent with all the other macros.

Replace _prop_rwlock_tryrdlock() by a macro _PROP_RWLOCK_TRYRDLOCK().
This avoids build failures in stand alone binaries (e.g. bootloaders)
where there is no "pa_rwlock" member in the "_prop_array" structure.

prop_object_iterator_next() and prop_object_iterator_reset() were
acquiring the rwlock recursively in some paths.

Introduce _prop_rwlock_tryrdlock() and use it in these functions, so
that the rwlock is *always* acquired once, while here add some
_PROP_RWLOCK_OWNED asserts to verify.

This definitely fixes the "locking against myself" panics.

Do not acquire the rwlock recursively in prop_{array,dictionary}_iterator_reset()
when prop_{array,dictionary}_copyout_ioctl() is called.

Introduce _PROP_RWLOCK_OWNED() which is a KASSERT(rw_lock_held(lock))
and use it in those two functions, also acquire the rwlock in other
places where it is required now.

This fixes a LOCKDEBUG panic "locking against myself", as reported by
Geoff C. Wing in current-users@.

Remove clause 3 and 4 from TNF licenses

branches: 1.23.2;
Actually implement prop_dictionary_make_immutable. OK'd by thorpej@.

branches: 1.22.2;
Include sys/simplelock.h in prop_object_impl.h.

sys/simplelock.h is required now.

Avoid using unbound amount of stack frames in prop_object_equal
by using a dynamic stack as well. Reorder arguments for the internalizer
as the iteration is always present and should go before possibly
NULL arguments.

Reviewed by mjf@ and adrianp@

branches: 1.19.2;
libprop is currently using a recursive parser. While this is fine for
userland, deeply nested arrays and dictionaries can easily overflow
the kernel stack and thereby force a panic.

Fix the internalizer and prop_object_release to use a separate call
stack and alter the dictionary and array handling to not recurse on
the C stack. The default stack has an inline depth of 16 elements,
which should keep the overhead reasonable.

This issue was found by Pavel Cahyna and Jachym Holecek.

Additionally add a limit for prop_object_copyin_ioctl to prevent user
programs from temporary allocating unbound amount of kernel memory.
Allow malloc to fail so that tight loops of userland processes can't
force panics by exhausting the kernel map.

Tested with the sample exploit of Jachym, his test suite and reviewed
by himself (initial patch), Christos Zoulas and Jason Thorpe.

boolean_t -> bool
TRUE -> true
FALSE -> false

prop_array_internalize and prop_dictionary_internalize are basically the
same code. Refactor it into _prop_generic_internalize, which gets passed
the second-level tag.

branches: 1.16.2; 1.16.4; 1.16.12;
- Add prop_dictionary_all_keys(). Takes a snapshot of a dictionary's
keys and returns them in an array. This effectively allows a caller
to mutate a dictionary while iterating over it (really, you iterate
over the array of keys instead of the dictionary itself).
- Add a slew of utility functions that make it more convenient (in some
circumstances, anyway) to get/set values in a dictionary.

Backout size_t casts - lint has been fixed.

Sprinkle a few size_t casts to avoid conversion warnings.

- Add a reader-writer lock to array and dictionary objects that ensures that
the data structure is internally consistent in the face of multiple threads
accessing it concurrently. This is not designed to provide application-
level semantic consistency; applications are responsible for that locking
protocol should it be necessary.
- Rename _PROP_MUTEX_DECL() to _PROP_MUTEX_DECL_STATIC().

Adapt Matt Thomas's red-black tree implementation for use in proplib. This
has a significant code size savings over <sys/tree.h>.

Also change prop_number_t to store all number objects in an r-b tree,
only ever allocating one object for any given number (we can do this
because numbers are immutable). This results in significant run-time
memory savings.

Also guard object accessors against NULL (like we do foreign object types)
so that apps can use this construct safely:

obj = prop_dictionary_get(dict, "value");
if (! prop_number_equals_integer(obj, 5)) {
...
}

Suggested by Iain Hibbert.

Add a workaround for GCC -Wuninitialized, similar to the prop_array.c
workaround, and properly marked with XXXGCC -Wuninitialized.

Several changes to proplib:
- Arrays can now be externalized and internalized in the same way
dictionaries can.
- Add new "externalize to file" and "internalize from file" functions
to make reading a property list from a file and writing a property
list to a file more convenient.
- Many assertions in the object implementations are gone. Instead,
calling an accessor for one object type with a different object type
as an argument will return a suitable "invalid" value.
- prop_object_type() now returns a new PROP_TYPE_UNKNOWN value if called
with a NULL object.
- Externalized property lists now contain a reference to the Apple XML
plist DTD.
- Add a new prop_ingest(3) facility, which provides a convenient way to
translate a dictionary into an arbitrary binary representation.

branches: 1.8.2;
Use a red-black tree to unique'ify the dictionary key symbols, rather than
a sorted array. Using a tree puts the linkage into the key symbol itself,
allowing us to avoid having to allocate memory with a mutex held.

remove unused variable that broke build

Keep a table of unique dictionary keysyms. This way, if we have 200 devices
with the "fw-path" property, we only have one copy of the "fw-path" string
as a dictionary key.

More cleanup.

Minor cleanup.

- Add prop_array_equals(), prop_dictionary_equals(), and
prop_dictionary_keysym_equals(), and prop_object_equals() functions.
- Use realloc() where it makes sense. There will be more changes in this
area.
- Add a _prop_object_type structure that is used internally to keep
information about the object types. Decreases the footprint of the
objects slightly by replacing several pointers with just one.

In _prop_dictionary_alloc(), explicitly set pd->pd_flags to zero
instead of leaving random garbage in there.
Should we just use "memset(pd, 0, sizeof(*pd));" instead?

Fixes boot problems on evbppc/WALNUT with new proplib.

Initial commit of proplib, a library for manipulating property lists and
converting to/from an XML external representation (based on Apple XML
property lists). Works in the kernel and user space.

branches: 1.5.2;
Add prop_array_t support to prop_kern.

Alter the protocol impemented by prop_kern to allow for bi-directional
plist-based messages and to eliminate looping previously required to
receive a plist from the kernel:

- prop_dictionary_copyin_ioctl() and prop_dictionary_copyout_ioctl()
now take the cmd argument rather than the file open flag. The
read-ness or write-ness of an ioctl command is checked by these
routines to ensure that information is being passed to/from the
userland component properly.

- prop_dictionary_copyout_ioctl() now allocates the memory for the
XML plist on behalf of the userland component by way of uvm_mmap().
The XML plist is copied out to the newly-mapped anonymous region,
and the pointer returned via the plistref.

- prop_dictionary_recv_ioctl() is responsible for munmap()'ing the
region after parsing the XML plist into internal represenatation.

- A new prop_dictionary_sendrecv_ioctl() is added, allowing user space
code to send a dictionary to the kernel and receive one back as a
reply.


Update users of prop_kern for the API changes (Bluetooth).

This constitutes an ABI / protocol change -- but this will also be put
into NetBSD 4.0 so that the first proplib release will implement the new
scheme.

Make HTML-safe.

Drop trailing whitespace.

Add kernel and user space parts of a protocol for sending property lists
to / from the kernel using ioctls.

branches: 1.5.2;
Add prop_array_t support to prop_kern.

Alter the protocol impemented by prop_kern to allow for bi-directional
plist-based messages and to eliminate looping previously required to
receive a plist from the kernel:

- prop_dictionary_copyin_ioctl() and prop_dictionary_copyout_ioctl()
now take the cmd argument rather than the file open flag. The
read-ness or write-ness of an ioctl command is checked by these
routines to ensure that information is being passed to/from the
userland component properly.

- prop_dictionary_copyout_ioctl() now allocates the memory for the
XML plist on behalf of the userland component by way of uvm_mmap().
The XML plist is copied out to the newly-mapped anonymous region,
and the pointer returned via the plistref.

- prop_dictionary_recv_ioctl() is responsible for munmap()'ing the
region after parsing the XML plist into internal represenatation.

- A new prop_dictionary_sendrecv_ioctl() is added, allowing user space
code to send a dictionary to the kernel and receive one back as a
reply.


Update users of prop_kern for the API changes (Bluetooth).

This constitutes an ABI / protocol change -- but this will also be put
into NetBSD 4.0 so that the first proplib release will implement the new
scheme.

Sort SEE ALSO, make HTML-safe.

Drop trailing whitespace.

Add kernel and user space parts of a protocol for sending property lists
to / from the kernel using ioctls.

Improvements to the problib(3) API:

==> Provide a much more complete set of setters and getters for different
value types in the prop_array_util(3) and prop_dictionary_util(3)
functions.

==> Overhaul the prop_data(3), prop_number(3), and prop_string(3) APIs
to be easier to use and less awkwardly named, Deprecate the old
awkward names, and produce link-time warnings when they are referenced.

==> Deprecate mutable prop_data(3) and prop_string(3) objects. The old
APIs that support them still exist, but will now produce link-time
warnings when used.

==> When the new prop_string(3) API is used, strings are internally
de-duplicated as a memory footprint optimization.

==> Provide a rich set of bounds-checked gettter functions in and a
corresponding set of convenience setters in the prop_number(3) API.

==> Add a new prop_bool_value(3) function that is equivalent to
prop_bool_true(3), but aligned with the new "value" routines in
prop_data(3), prop_string(3), and prop_number(3).

branches: 1.9.12;
Use .Fn to refer a function name instead of Nm.
Also, use Xr to refer problib(3) in the HISTORY section instead of Nm.

While parsing the man pages, any .Nm occurrence gets replaced by the
value specified in the NAME section. Referencing individual function
names with .Nm was causing makemandb(8) to replicate the complete
NAME section for every such occurrence. This was leading to an
ugly looking snippet in apropos(1)'s output when searched for
any of these man pages.

branches: 1.8.26; 1.8.30;
Add missing Nd.

prop_dictionary_get_dict: last argument is prop_dictionary_t *, not bool *

Fix a typo and a punctuation nit.

Add a new libquota library, which contains some blocks to build and/or
parse quota plists; as well as a getfsquota() function to retrieve quotas
for a single id from a single filesystem (whatever filesystem this is:
a local quota-enabled fs or NFS). This is build on functions getufsquota()
(for local filesystems with UFS-like quotas) and getnfsquota();
which are also available to userland programs.
move functions from quota2_subr.c to libquota or libprop as appropriate,
and ajust in-tree quota tools.
move some declarations from kernel headers to either sys/quota.h or
quota/quota.h as appropriate. ufs/ufs/quota.h still installed because
it's needed by other installed ufs headers.
ufs/ufs/quota1.h still installed as a quick&dirty way to get a code
using the old quotactl() to compile (just include ufs/ufs/quota1.h instead of
ufs/ufs/quota.h - old code won't compile without this change and this is
on purpose).
Discussed on tech-kern@ and tech-net@ (long thread, but not much about
libquota itself ...)

Fix typo.

OK by martin@.

branches: 1.3.2;
Convert TNF licenses to new 2 clause variant

branches: 1.2.8;
boolean_t -> bool
TRUE -> true
FALSE -> false

branches: 1.1.2; 1.1.4; 1.1.16;
- Add prop_dictionary_all_keys(). Takes a snapshot of a dictionary's
keys and returns them in an array. This effectively allows a caller
to mutate a dictionary while iterating over it (really, you iterate
over the array of keys instead of the dictionary itself).
- Add a slew of utility functions that make it more convenient (in some
circumstances, anyway) to get/set values in a dictionary.

Hide the private symbols.

branches: 1.9.8;
proplib: Nix trailing whitespace.

remove error(1) comments

add/fix linted comments

Improvements to the problib(3) API:

==> Provide a much more complete set of setters and getters for different
value types in the prop_array_util(3) and prop_dictionary_util(3)
functions.

==> Overhaul the prop_data(3), prop_number(3), and prop_string(3) APIs
to be easier to use and less awkwardly named, Deprecate the old
awkward names, and produce link-time warnings when they are referenced.

==> Deprecate mutable prop_data(3) and prop_string(3) objects. The old
APIs that support them still exist, but will now produce link-time
warnings when used.

==> When the new prop_string(3) API is used, strings are internally
de-duplicated as a memory footprint optimization.

==> Provide a rich set of bounds-checked gettter functions in and a
corresponding set of convenience setters in the prop_number(3) API.

==> Add a new prop_bool_value(3) function that is equivalent to
prop_bool_true(3), but aligned with the new "value" routines in
prop_data(3), prop_string(3), and prop_number(3).

branches: 1.5.34;
Bit-un-rot the proplib portability layer.

branches: 1.4.4;
Add a new libquota library, which contains some blocks to build and/or
parse quota plists; as well as a getfsquota() function to retrieve quotas
for a single id from a single filesystem (whatever filesystem this is:
a local quota-enabled fs or NFS). This is build on functions getufsquota()
(for local filesystems with UFS-like quotas) and getnfsquota();
which are also available to userland programs.
move functions from quota2_subr.c to libquota or libprop as appropriate,
and ajust in-tree quota tools.
move some declarations from kernel headers to either sys/quota.h or
quota/quota.h as appropriate. ufs/ufs/quota.h still installed because
it's needed by other installed ufs headers.
ufs/ufs/quota1.h still installed as a quick&dirty way to get a code
using the old quotactl() to compile (just include ufs/ufs/quota1.h instead of
ufs/ufs/quota.h - old code won't compile without this change and this is
on purpose).
Discussed on tech-kern@ and tech-net@ (long thread, but not much about
libquota itself ...)

Remove clause 3 and 4 from TNF licenses

branches: 1.2.8;
boolean_t -> bool
TRUE -> true
FALSE -> false

branches: 1.1.2; 1.1.4; 1.1.12;
- Add prop_dictionary_all_keys(). Takes a snapshot of a dictionary's
keys and returns them in an array. This effectively allows a caller
to mutate a dictionary while iterating over it (really, you iterate
over the array of keys instead of the dictionary itself).
- Add a slew of utility functions that make it more convenient (in some
circumstances, anyway) to get/set values in a dictionary.

Fix various typos, mainly in comments.

branches: 1.2.4;
Undo previous revert, and fix include ordering in prop_extern.c and
prop_intern.c that broke tools build on some hosts.

Separate the {extern,intern}alize support routines into their own files,
and reduce the verbosity of the names.

NFCI.

branches: 1.8.12;
Add entry for prop_ingest in the NAME section.

Use .Fn to refer a function name instead of Nm.
Also, use Xr to refer problib(3) in the HISTORY section instead of Nm.

While parsing the man pages, any .Nm occurrence gets replaced by the
value specified in the NAME section. Referencing individual function
names with .Nm was causing makemandb(8) to replicate the complete
NAME section for every such occurrence. This was leading to an
ugly looking snippet in apropos(1)'s output when searched for
any of these man pages.

branches: 1.6.30; 1.6.34;
Fix typo (specifed -> specified).

Convert TNF licenses to new 2 clause variant

branches: 1.4.4;
PR/37838: Stathis Kamperis: proplib documentation enhancements

branches: 1.3.2;
boolean_t -> bool
TRUE -> true
FALSE -> false

branches: 1.2.2; 1.2.6; 1.2.18;
Remove duplicate word. Markup fixes.

Several changes to proplib:
- Arrays can now be externalized and internalized in the same way
dictionaries can.
- Add new "externalize to file" and "internalize from file" functions
to make reading a property list from a file and writing a property
list to a file more convenient.
- Many assertions in the object implementations are gone. Instead,
calling an accessor for one object type with a different object type
as an argument will return a suitable "invalid" value.
- prop_object_type() now returns a new PROP_TYPE_UNKNOWN value if called
with a NULL object.
- Externalized property lists now contain a reference to the Apple XML
plist DTD.
- Add a new prop_ingest(3) facility, which provides a convenient way to
translate a dictionary into an arbitrary binary representation.

Hide the private symbols.

branches: 1.5.16; 1.5.36;
Eliminate use of C++ keywords and don't nest struct definitions.

Bit-un-rot the proplib portability layer.

branches: 1.3.4;
Remove clause 3 and 4 from TNF licenses

branches: 1.2.8;
boolean_t -> bool
TRUE -> true
FALSE -> false

branches: 1.1.2; 1.1.6; 1.1.14;
Several changes to proplib:
- Arrays can now be externalized and internalized in the same way
dictionaries can.
- Add new "externalize to file" and "internalize from file" functions
to make reading a property list from a file and writing a property
list to a file more convenient.
- Many assertions in the object implementations are gone. Instead,
calling an accessor for one object type with a different object type
as an argument will return a suitable "invalid" value.
- prop_object_type() now returns a new PROP_TYPE_UNKNOWN value if called
with a NULL object.
- Externalized property lists now contain a reference to the Apple XML
plist DTD.
- Add a new prop_ingest(3) facility, which provides a convenient way to
translate a dictionary into an arbitrary binary representation.

branches: 1.2.4;
Undo previous revert, and fix include ordering in prop_extern.c and
prop_intern.c that broke tools build on some hosts.

Separate the {extern,intern}alize support routines into their own files,
and reduce the verbosity of the names.

NFCI.

Undo the last change (rev 1.29) and instead elide the entire contents
of this file if HAVE_NBTOOL_CONFIG_H is defined -- what happens in this
file is only the business of the target system and of no interest to
build tools, regardless of which OS those tools run upon.

Fix builds on NetBSD hosts.

Do not #include prop_object_impl.h in tools or _STANDALONE
builds, it (directly or indirectly) brings in far too much
that is inappropriate in those environments.

If its sole reason for being added was for the fallback
definition of _PROP_EXPORT then in those cases, simply
supply that here.

This is most probably not the most elegant of fixes, and a
better way of supplying that fallback definition than dumping
it in the middle of prop_object_impl.h should be found.

Someone else's problem - but DO test any alternative version
of this with a complete clean build on a NetBSD host before
committing. And note, that since prop_object_impl.h includes
nbtool_config.h, when appropriate, it MUST be included before
any other system include file (but that change alone is not
sufficient to fix things, so do it this way instead).

This should unbreak all (well, all the previously unbroken)
builds on NetBSD hosts.

Hide the private symbols.

Add support for serializing to, and de-serializing from, JSON, as
specified in RFC 8259. Format is automatically detected for input,
and can be specified explicitly for output. The default format remains
XML.

As part of this change, generic prop_object_{internalize,externalize}()
functions are provided, and the "_with_format" flavor allows the output
format to be specified. Previous array- and dictionary-specific routines
remain for backwards compatibility.

fix a/an grammar in obvious cases

branches: 1.25.8;
proplib: Nix trailing whitespace.

remove M_CANFAIL flag for malloc(9) - it was completely ignored, so had
actually no effect

branches: 1.23.10; 1.23.12;
call the proper size functions

missing brace

add sized versions of the copyin ioctls

branches: 1.20.2;
bump to 128K (because of npf large rules) and return E2BIG.

branches: 1.19.2;
Limit size of xml buffer for userland requests (From Mateusz Kocielski)
XXX: pullup-7

add a new "fo_mmap" fileops method to allow use of arbitrary uvm_objects for
mappings of file objects. move vnode-specific details of mmap()ing a vnode
from uvm_mmap() to the new vnode-specific vn_mmap(). add new uvm_mmap_dev()
and uvm_mmap_anon() convenience functions for mapping character devices
and anonymous memory, and replace all other calls to uvm_mmap() with those.
use the new fileop in drm2 so that libdrm can use mmap() to map things
like on other platforms (instead of the ioctl that we have used so far).

branches: 1.17.22;
Fix an implementation inconsistency with the prop_*_send_syscall() and
prop_*_recv_syscall() functions from proplib(3). They now share the
same logic as the one from prop_*_send_ioctl() functions:
- returns an int
- 0 indicates "no error", otherwise returns the error number (and
set errno)

Many consumers of the prop_*_{ioctl, syscall} expect errno to be set
on error and use err() to display the error message. As such, ensures that
errno gets set before returning from these functions.

prop_*_send_syscall() functions returned a boolean, and now return an int.
Fix all call sites to use the new paradigm (only quota2 is affected in src).

As the prop_*_{send,recv}_syscall() API appeared in -current and is only
used by the recent quota2 code, I am not bumping the lib. The API change
only affects the prop_*_send_syscall() function (recv_syscall()s were
already used correctly), so ensure you are not mixing "old" -current
quota binaries with a new proplib(3) (or the other way around). This
change will be announced via a HEADS-UP and UPDATING.

Does not affect the kernel part of proplib.

Document the correct API in prop_array(3) and prop_dictionary(3).

Thanks to Francois Tigeot for noticing the API inconsistency and
reporting it on tech-kern@.

ok bouyer@.

prop_*_copyout takes an object as second parameter, not a pointer to object.

- Implement prop_{array,dictionary}_copyout(), based on
prop_{array,dictionary}_copyout_ioctl().
- Implement prop_{array,dictionary}_recv_syscall(), which
takes as parameter the pref we got from kernel and internalize it,
and unmaps the buffer prop_{array,dictionary}_copyout() mapped for us.
- add a prop_{array,dictionary}_send_syscall() for symetry, which is
an alias to prop_{array,dictionary}_externalize_to_pref()

Discussed on tech-kern@ and tech-userlevel@

Use a consistent approach for rump kernel calls made by libs.
Should investigate a dynamic approach in the future.

Back out previous. Builds for me and on autobuild cluster.

make this compile again.

Implement prop_array_externalize_to_pref(), prop_array_copyin(),
prop_dictionary_externalize_to_pref() and prop_dictionary_copyin()
as discussed in the thread:
http://mail-index.netbsd.org/tech-kern/2009/07/29/msg005594.html

Bump libprop shlib minor.

Wipe out the last vestiges of POOL_INIT with one swift stroke. In
most cases, use a proper constructor. For proplib, give a local
equivalent of POOL_INIT for the kernel object implementation. This
way the code structure can be preserved, and a local link set is
not hazardous anyway (unless proplib is split to several modules,
but that'll be the day).

tested by booting a kernel in qemu and compile-testing i386/ALL

Remove clause 3 and 4 from TNF licenses

branches: 1.8.8;
libprop is currently using a recursive parser. While this is fine for
userland, deeply nested arrays and dictionaries can easily overflow
the kernel stack and thereby force a panic.

Fix the internalizer and prop_object_release to use a separate call
stack and alter the dictionary and array handling to not recurse on
the C stack. The default stack has an inline depth of 16 elements,
which should keep the overhead reasonable.

This issue was found by Pavel Cahyna and Jachym Holecek.

Additionally add a limit for prop_object_copyin_ioctl to prevent user
programs from temporary allocating unbound amount of kernel memory.
Allow malloc to fail so that tight loops of userland processes can't
force panics by exhausting the kernel map.

Tested with the sample exploit of Jachym, his test suite and reviewed
by himself (initial patch), Christos Zoulas and Jason Thorpe.

branches: 1.7.2; 1.7.10;
Add a comment to explain how the value for uaddr is chosen. Maybe uvm_mmap
should do that instead of sys_mmap...

In _prop_object_copyout_ioctl, uaddr was not initialised, although it is
used by uvm_mmap() as a hint for the virtual address to map memory to. As
a consequence, it tended to fail a lot on some architectures.

We cannot use 0, so instead use the value that would have been used if we
were calling mmap(2) with 0 as the first argument.

Fixes PR#34639 by xtraeme@.

branches: 1.5.2;
Add prop_array_t support to prop_kern.

unbreak kernel builds from the last fix

protect include of sys/mman.h when building _KERNEL or _STANDALONE

Alter the protocol impemented by prop_kern to allow for bi-directional
plist-based messages and to eliminate looping previously required to
receive a plist from the kernel:

- prop_dictionary_copyin_ioctl() and prop_dictionary_copyout_ioctl()
now take the cmd argument rather than the file open flag. The
read-ness or write-ness of an ioctl command is checked by these
routines to ensure that information is being passed to/from the
userland component properly.

- prop_dictionary_copyout_ioctl() now allocates the memory for the
XML plist on behalf of the userland component by way of uvm_mmap().
The XML plist is copied out to the newly-mapped anonymous region,
and the pointer returned via the plistref.

- prop_dictionary_recv_ioctl() is responsible for munmap()'ing the
region after parsing the XML plist into internal represenatation.

- A new prop_dictionary_sendrecv_ioctl() is added, allowing user space
code to send a dictionary to the kernel and receive one back as a
reply.


Update users of prop_kern for the API changes (Bluetooth).

This constitutes an ABI / protocol change -- but this will also be put
into NetBSD 4.0 so that the first proplib release will implement the new
scheme.

Add kernel and user space parts of a protocol for sending property lists
to / from the kernel using ioctls.

Improvements to the problib(3) API:

==> Provide a much more complete set of setters and getters for different
value types in the prop_array_util(3) and prop_dictionary_util(3)
functions.

==> Overhaul the prop_data(3), prop_number(3), and prop_string(3) APIs
to be easier to use and less awkwardly named, Deprecate the old
awkward names, and produce link-time warnings when they are referenced.

==> Deprecate mutable prop_data(3) and prop_string(3) objects. The old
APIs that support them still exist, but will now produce link-time
warnings when used.

==> When the new prop_string(3) API is used, strings are internally
de-duplicated as a memory footprint optimization.

==> Provide a rich set of bounds-checked gettter functions in and a
corresponding set of convenience setters in the prop_number(3) API.

==> Add a new prop_bool_value(3) function that is equivalent to
prop_bool_true(3), but aligned with the new "value" routines in
prop_data(3), prop_string(3), and prop_number(3).

branches: 1.11.12;
Use .Fn to refer a function name instead of Nm.
Also, use Xr to refer problib(3) in the HISTORY section instead of Nm.

While parsing the man pages, any .Nm occurrence gets replaced by the
value specified in the NAME section. Referencing individual function
names with .Nm was causing makemandb(8) to replicate the complete
NAME section for every such occurrence. This was leading to an
ugly looking snippet in apropos(1)'s output when searched for
any of these man pages.

branches: 1.10.30; 1.10.34;
New sentence, new line. Use Dq.

Convert TNF licenses to new 2 clause variant

branches: 1.8.4;
PR/37838: Stathis Kamperis: proplib documentation enhancements

branches: 1.7.2;
boolean_t -> bool
TRUE -> true
FALSE -> false

Fix typo.

branches: 1.5.2; 1.5.4; 1.5.8;
Put macro argument right after macro, not in next line.

Make prop_number_t handle both signed and unsigned numbers. The *integer*
routines now take int64_t arguments, and new *unsigned_integer* routines
take uint64_t arguments. See prop_number(3) for complete details.

Drop trailing whitespace.

Change prop_number_t to store a uint64_t, rather than a uintmax_t. No
ABI change, and we ought to explicitly state that this is a fixed-width
type, since that what it needs to be for binary plist support.

Initial commit of proplib, a library for manipulating property lists and
converting to/from an XML external representation (based on Apple XML
property lists). Works in the kernel and user space.

Undo previous revert, and fix include ordering in prop_extern.c and
prop_intern.c that broke tools build on some hosts.

Revert previous while I investigate the build failure.

Separate the {extern,intern}alize support routines into their own files,
and reduce the verbosity of the names.

NFCI.

Hide the private symbols.

Add support for serializing to, and de-serializing from, JSON, as
specified in RFC 8259. Format is automatically detected for input,
and can be specified explicitly for output. The default format remains
XML.

As part of this change, generic prop_object_{internalize,externalize}()
functions are provided, and the "_with_format" flavor allows the output
format to be specified. Previous array- and dictionary-specific routines
remain for backwards compatibility.

branches: 1.34.8;
proplib: Nix trailing whitespace.

Correct a deprecation warning.

Improvements to the problib(3) API:

==> Provide a much more complete set of setters and getters for different
value types in the prop_array_util(3) and prop_dictionary_util(3)
functions.

==> Overhaul the prop_data(3), prop_number(3), and prop_string(3) APIs
to be easier to use and less awkwardly named, Deprecate the old
awkward names, and produce link-time warnings when they are referenced.

==> Deprecate mutable prop_data(3) and prop_string(3) objects. The old
APIs that support them still exist, but will now produce link-time
warnings when used.

==> When the new prop_string(3) API is used, strings are internally
de-duplicated as a memory footprint optimization.

==> Provide a rich set of bounds-checked gettter functions in and a
corresponding set of convenience setters in the prop_number(3) API.

==> Add a new prop_bool_value(3) function that is equivalent to
prop_bool_true(3), but aligned with the new "value" routines in
prop_data(3), prop_string(3), and prop_number(3).

Fix building libprop as a host tool library on platforms that don't have
the Matt Thomas rbtree:

- Include rb.c in libnbcompat, and provide a nbcompat sys/rbtree.h
header.
- Make sure libprop's source file include prop_object_impl.h before
anything else, and pull in nbtool_config.h from there.

Tested by simulating such a host system by renaming the host's
<sys/rbtree.h> out of the way (which reproduced the build failure)
and verifying that the host-tool installboot contained the rb_*
functions in its own .text segment.

branches: 1.30.16;
Fix the previous fix - there's only one _node suffix here!

Missed a couple of function-call renames in previous

The local copy of rb_tree code has been unused for quite some time. So
we can just remove it, and adjust callers to use the "real" rbtree
function names.

Addresses PR lib/44090

Eliminate use of C++ keywords and don't nest struct definitions.

kill sprintf

Make this compilable with gcc 4.8.1 without options DIAGNOSTIC.

branches: 1.24.2;
Bit-un-rot the proplib portability layer.

branches: 1.23.6;
Fixes/improvements to RB-tree implementation:
1. Fix inverted node order, so that negative value from comparison operator
would represent lower (left) node, and positive - higher (right) node.
2. Add an argument (i.e. "context"), passed to comparison operators.
3. Change rb_tree_insert_node() to return a node - either inserted one or
already existing one.
4. Amend the interface to manipulate the actual object, instead of the
rb_node (in a similar way as Patricia-tree interface does).
5. Update all RB-tree users accordingly.

XXX: Perhaps rename rb.h to rbtree.h, since cleaning-up..

1-3 address the PR/43488 by Jeremy Huddleston.

Passes RB-tree regression tests.
Reviewed by: matt@, christos@

ansify function definitions

branches: 1.21.2;
Get rid of locks with static initializers using once and atomic ops.
This makes proplib simplelock-free.

Fix two race conditions in proplib library. In prop_dictionary and prop_number
there was small window in which was entry left in rb tree with reference
count 0 which lead to rb tree coruption when another thread picked this up
before it was released.
Add 2 new members to the prop_object_t which are used for locking/unlocking
rb tree guard mutex.

Ok by joerg@, core@, thorpej@

branches: 1.19.4;
Fix an issue introduced with the prop_stack change: It's not a good idea to
return a 3-value enum from a function declared to return bool. This broke
the recurse case for prop_object_equals(). Instead, declare the object type
equals routine to return a _prop_object_equals_rv_t.

Give the same treatment to the object type free routines: declare them to
return a _prop_object_free_rv_t, and consistently check those return values
againt the enum type.

Tidy up some whitespace while we're here.

_prop_rb_tree_insert_node() now returns true/false to indicate if the
insertion succeeded. Update existing usage that arranges for insertions
to always succeed to assert that they do.

branches: 1.17.2;
Remove clause 3 and 4 from TNF licenses

branches: 1.16.4;
Include sys/simplelock.h in prop_object_impl.h.

sys/simplelock.h is required now.

Avoid using unbound amount of stack frames in prop_object_equal
by using a dynamic stack as well. Reorder arguments for the internalizer
as the iteration is always present and should go before possibly
NULL arguments.

Reviewed by mjf@ and adrianp@

branches: 1.13.2;
libprop is currently using a recursive parser. While this is fine for
userland, deeply nested arrays and dictionaries can easily overflow
the kernel stack and thereby force a panic.

Fix the internalizer and prop_object_release to use a separate call
stack and alter the dictionary and array handling to not recurse on
the C stack. The default stack has an inline depth of 16 elements,
which should keep the overhead reasonable.

This issue was found by Pavel Cahyna and Jachym Holecek.

Additionally add a limit for prop_object_copyin_ioctl to prevent user
programs from temporary allocating unbound amount of kernel memory.
Allow malloc to fail so that tight loops of userland processes can't
force panics by exhausting the kernel map.

Tested with the sample exploit of Jachym, his test suite and reviewed
by himself (initial patch), Christos Zoulas and Jason Thorpe.

boolean_t -> bool
TRUE -> true
FALSE -> false

branches: 1.11.2; 1.11.4; 1.11.12;
delete junk I accidentally committed.

de-lint.

Appease our stupid lint.

Make prop_number_t handle both signed and unsigned numbers. The *integer*
routines now take int64_t arguments, and new *unsigned_integer* routines
take uint64_t arguments. See prop_number(3) for complete details.

- Add a reader-writer lock to array and dictionary objects that ensures that
the data structure is internally consistent in the face of multiple threads
accessing it concurrently. This is not designed to provide application-
level semantic consistency; applications are responsible for that locking
protocol should it be necessary.
- Rename _PROP_MUTEX_DECL() to _PROP_MUTEX_DECL_STATIC().

Adapt Matt Thomas's red-black tree implementation for use in proplib. This
has a significant code size savings over <sys/tree.h>.

Also change prop_number_t to store all number objects in an r-b tree,
only ever allocating one object for any given number (we can do this
because numbers are immutable). This results in significant run-time
memory savings.

Also guard object accessors against NULL (like we do foreign object types)
so that apps can use this construct safely:

obj = prop_dictionary_get(dict, "value");
if (! prop_number_equals_integer(obj, 5)) {
...
}

Suggested by Iain Hibbert.

Several changes to proplib:
- Arrays can now be externalized and internalized in the same way
dictionaries can.
- Add new "externalize to file" and "internalize from file" functions
to make reading a property list from a file and writing a property
list to a file more convenient.
- Many assertions in the object implementations are gone. Instead,
calling an accessor for one object type with a different object type
as an argument will return a suitable "invalid" value.
- prop_object_type() now returns a new PROP_TYPE_UNKNOWN value if called
with a NULL object.
- Externalized property lists now contain a reference to the Apple XML
plist DTD.
- Add a new prop_ingest(3) facility, which provides a convenient way to
translate a dictionary into an arbitrary binary representation.

branches: 1.3.2;
Change prop_number_t to store a uint64_t, rather than a uintmax_t. No
ABI change, and we ought to explicitly state that this is a fixed-width
type, since that what it needs to be for binary plist support.

- Add prop_array_equals(), prop_dictionary_equals(), and
prop_dictionary_keysym_equals(), and prop_object_equals() functions.
- Use realloc() where it makes sense. There will be more changes in this
area.
- Add a _prop_object_type structure that is used internally to keep
information about the object types. Decreases the footprint of the
objects slightly by replacing several pointers with just one.

Initial commit of proplib, a library for manipulating property lists and
converting to/from an XML external representation (based on Apple XML
property lists). Works in the kernel and user space.

Add support for serializing to, and de-serializing from, JSON, as
specified in RFC 8259. Format is automatically detected for input,
and can be specified explicitly for output. The default format remains
XML.

As part of this change, generic prop_object_{internalize,externalize}()
functions are provided, and the "_with_format" flavor allows the output
format to be specified. Previous array- and dictionary-specific routines
remain for backwards compatibility.

branches: 1.10.12;
Improvements to the problib(3) API:

==> Provide a much more complete set of setters and getters for different
value types in the prop_array_util(3) and prop_dictionary_util(3)
functions.

==> Overhaul the prop_data(3), prop_number(3), and prop_string(3) APIs
to be easier to use and less awkwardly named, Deprecate the old
awkward names, and produce link-time warnings when they are referenced.

==> Deprecate mutable prop_data(3) and prop_string(3) objects. The old
APIs that support them still exist, but will now produce link-time
warnings when used.

==> When the new prop_string(3) API is used, strings are internally
de-duplicated as a memory footprint optimization.

==> Provide a rich set of bounds-checked gettter functions in and a
corresponding set of convenience setters in the prop_number(3) API.

==> Add a new prop_bool_value(3) function that is equivalent to
prop_bool_true(3), but aligned with the new "value" routines in
prop_data(3), prop_string(3), and prop_number(3).

branches: 1.9.12;
Use .Fn to refer a function name instead of Nm.
Also, use Xr to refer problib(3) in the HISTORY section instead of Nm.

While parsing the man pages, any .Nm occurrence gets replaced by the
value specified in the NAME section. Referencing individual function
names with .Nm was causing makemandb(8) to replicate the complete
NAME section for every such occurrence. This was leading to an
ugly looking snippet in apropos(1)'s output when searched for
any of these man pages.

branches: 1.8.30; 1.8.34;
New sentence, new line. Use Dq.

Convert TNF licenses to new 2 clause variant

branches: 1.6.8;
boolean_t -> bool
TRUE -> true
FALSE -> false

branches: 1.5.4; 1.5.16;
Bump date for previous.

Several changes to proplib:
- Arrays can now be externalized and internalized in the same way
dictionaries can.
- Add new "externalize to file" and "internalize from file" functions
to make reading a property list from a file and writing a property
list to a file more convenient.
- Many assertions in the object implementations are gone. Instead,
calling an accessor for one object type with a different object type
as an argument will return a suitable "invalid" value.
- prop_object_type() now returns a new PROP_TYPE_UNKNOWN value if called
with a NULL object.
- Externalized property lists now contain a reference to the Apple XML
plist DTD.
- Add a new prop_ingest(3) facility, which provides a convenient way to
translate a dictionary into an arbitrary binary representation.

branches: 1.3.2;
Drop trailing whitespace.

- Add prop_array_equals(), prop_dictionary_equals(), and
prop_dictionary_keysym_equals(), and prop_object_equals() functions.
- Use realloc() where it makes sense. There will be more changes in this
area.
- Add a _prop_object_type structure that is used internally to keep
information about the object types. Decreases the footprint of the
objects slightly by replacing several pointers with just one.

Initial commit of proplib, a library for manipulating property lists and
converting to/from an XML external representation (based on Apple XML
property lists). Works in the kernel and user space.

Undo previous revert, and fix include ordering in prop_extern.c and
prop_intern.c that broke tools build on some hosts.

Revert previous while I investigate the build failure.

Separate the {extern,intern}alize support routines into their own files,
and reduce the verbosity of the names.

NFCI.

Hide the private symbols.

pass lint

typo police.

Add support for serializing to, and de-serializing from, JSON, as
specified in RFC 8259. Format is automatically detected for input,
and can be specified explicitly for output. The default format remains
XML.

As part of this change, generic prop_object_{internalize,externalize}()
functions are provided, and the "_with_format" flavor allows the output
format to be specified. Previous array- and dictionary-specific routines
remain for backwards compatibility.

branches: 1.35.8;
proplib: Allocate sizeof(*ctx), not sizeof(struct ...).

No functional change intended.

proplib: Fix mistake in previous -- use strncmp for prefix matching.

proplib: Don't run off end of buffer with memcmp.

The input is required to be NUL-terminated anyway, so just use strcmp
here.

Reported-by: syzbot+69838802c8ec55909ba3@syzkaller.appspotmail.com
https://syzkaller.appspot.com/bug?id=927d66e8aa079ba2be43497425a6d9878025ad09

proplib: Nix trailing whitespace.

Use posix_madvise() rather than the legacy madvise() call, and
wrapp the calls in #ifdef for the advice we're giving. Should
address reports of host tool build issues.

branches: 1.30.16;
Now that _PROP_ISSPACE does not include the EOF check, put the check for
EOF inside the loop. Also fix another unbounded loop that did not check for
EOF. From Mateusz Kocielski
XXX: pullup-7

branches: 1.29.4;
Make this compilable with gcc 4.8.1 without options DIAGNOSTIC.

branches: 1.28.2;
Bit-un-rot the proplib portability layer.

branches: 1.27.4;
Update also the non-void pointers to the current test objects.
Finaly fixes PR lib/43964.

Fix crash where user was able to crash proplib with trying to internalize
bad xml file with non-existing data type e.g. <number>.

Problem is that poi is not NULL even in case that we haven't find any match
in data type name. We need to check if poi->poi_tag is not NULL before
calling poi->poi_intern function which is non existing case NULL and will
cause crash.

branches: 1.25.2;
sign-compare fix

Get rid of locks with static initializers using once and atomic ops.
This makes proplib simplelock-free.

Fix two race conditions in proplib library. In prop_dictionary and prop_number
there was small window in which was entry left in rb tree with reference
count 0 which lead to rb tree coruption when another thread picked this up
before it was released.
Add 2 new members to the prop_object_t which are used for locking/unlocking
rb tree guard mutex.

Ok by joerg@, core@, thorpej@

branches: 1.22.4;
Fix an issue introduced with the prop_stack change: It's not a good idea to
return a 3-value enum from a function declared to return bool. This broke
the recurse case for prop_object_equals(). Instead, declare the object type
equals routine to return a _prop_object_equals_rv_t.

Give the same treatment to the object type free routines: declare them to
return a _prop_object_free_rv_t, and consistently check those return values
againt the enum type.

Tidy up some whitespace while we're here.

branches: 1.21.2;
Remove clause 3 and 4 from TNF licenses

branches: 1.20.2;
Use mkstemp() + fchmod() instead of mktemp(), since the latter
results in linker warnings.

branches: 1.19.2;
Include sys/simplelock.h in prop_object_impl.h.

sys/simplelock.h is required now.

tyop: guaranty -> guarantee.

Avoid using unbound amount of stack frames in prop_object_equal
by using a dynamic stack as well. Reorder arguments for the internalizer
as the iteration is always present and should go before possibly
NULL arguments.

Reviewed by mjf@ and adrianp@

branches: 1.15.2;
libprop is currently using a recursive parser. While this is fine for
userland, deeply nested arrays and dictionaries can easily overflow
the kernel stack and thereby force a panic.

Fix the internalizer and prop_object_release to use a separate call
stack and alter the dictionary and array handling to not recurse on
the C stack. The default stack has an inline depth of 16 elements,
which should keep the overhead reasonable.

This issue was found by Pavel Cahyna and Jachym Holecek.

Additionally add a limit for prop_object_copyin_ioctl to prevent user
programs from temporary allocating unbound amount of kernel memory.
Allow malloc to fail so that tight loops of userland processes can't
force panics by exhausting the kernel map.

Tested with the sample exploit of Jachym, his test suite and reviewed
by himself (initial patch), Christos Zoulas and Jason Thorpe.

boolean_t -> bool
TRUE -> true
FALSE -> false

prop_array_internalize and prop_dictionary_internalize are basically the
same code. Refactor it into _prop_generic_internalize, which gets passed
the second-level tag.

branches: 1.12.2; 1.12.4; 1.12.12;
sysconf() returns long, which isn't neccessarily assignment-compatible
with size_t, as evidenced by sh5, so add an explicit cast.

Backout size_t casts - lint has been fixed.

Sprinkle a few size_t casts to avoid conversion warnings.

Add a _PROP_ARG_UNUSED that expands to __unused on NetBSD and to nothing
everywhere else. Use it where Christos decided to use __unused in this
code.

de-lint.

_prop_object_fini(): Consume all of the arguments.

- Add a reader-writer lock to array and dictionary objects that ensures that
the data structure is internally consistent in the face of multiple threads
accessing it concurrently. This is not designed to provide application-
level semantic consistency; applications are responsible for that locking
protocol should it be necessary.
- Rename _PROP_MUTEX_DECL() to _PROP_MUTEX_DECL_STATIC().

complete initialization.

Several changes to proplib:
- Arrays can now be externalized and internalized in the same way
dictionaries can.
- Add new "externalize to file" and "internalize from file" functions
to make reading a property list from a file and writing a property
list to a file more convenient.
- Many assertions in the object implementations are gone. Instead,
calling an accessor for one object type with a different object type
as an argument will return a suitable "invalid" value.
- prop_object_type() now returns a new PROP_TYPE_UNKNOWN value if called
with a NULL object.
- Externalized property lists now contain a reference to the Apple XML
plist DTD.
- Add a new prop_ingest(3) facility, which provides a convenient way to
translate a dictionary into an arbitrary binary representation.

branches: 1.3.2;
Move all of the locking compatibility definitions into prop_object_impl.h.

- Add prop_array_equals(), prop_dictionary_equals(), and
prop_dictionary_keysym_equals(), and prop_object_equals() functions.
- Use realloc() where it makes sense. There will be more changes in this
area.
- Add a _prop_object_type structure that is used internally to keep
information about the object types. Decreases the footprint of the
objects slightly by replacing several pointers with just one.

Initial commit of proplib, a library for manipulating property lists and
converting to/from an XML external representation (based on Apple XML
property lists). Works in the kernel and user space.

Undo previous revert, and fix include ordering in prop_extern.c and
prop_intern.c that broke tools build on some hosts.

Revert previous while I investigate the build failure.

Separate the {extern,intern}alize support routines into their own files,
and reduce the verbosity of the names.

NFCI.

Hide the private symbols.

Add support for serializing to, and de-serializing from, JSON, as
specified in RFC 8259. Format is automatically detected for input,
and can be specified explicitly for output. The default format remains
XML.

As part of this change, generic prop_object_{internalize,externalize}()
functions are provided, and the "_with_format" flavor allows the output
format to be specified. Previous array- and dictionary-specific routines
remain for backwards compatibility.

branches: 1.36.12;
Change previous to only apply when building with clang.

Unbreak clang builds by removing questionable linker warning sections
trggered all over the place.

Improvements to the problib(3) API:

==> Provide a much more complete set of setters and getters for different
value types in the prop_array_util(3) and prop_dictionary_util(3)
functions.

==> Overhaul the prop_data(3), prop_number(3), and prop_string(3) APIs
to be easier to use and less awkwardly named, Deprecate the old
awkward names, and produce link-time warnings when they are referenced.

==> Deprecate mutable prop_data(3) and prop_string(3) objects. The old
APIs that support them still exist, but will now produce link-time
warnings when used.

==> When the new prop_string(3) API is used, strings are internally
de-duplicated as a memory footprint optimization.

==> Provide a rich set of bounds-checked gettter functions in and a
corresponding set of convenience setters in the prop_number(3) API.

==> Add a new prop_bool_value(3) function that is equivalent to
prop_bool_true(3), but aligned with the new "value" routines in
prop_data(3), prop_string(3), and prop_number(3).

Fix building libprop as a host tool library on platforms that don't have
the Matt Thomas rbtree:

- Include rb.c in libnbcompat, and provide a nbcompat sys/rbtree.h
header.
- Make sure libprop's source file include prop_object_impl.h before
anything else, and pull in nbtool_config.h from there.

Tested by simulating such a host system by renaming the host's
<sys/rbtree.h> out of the way (which reproduced the build failure)
and verifying that the host-tool installboot contained the rb_*
functions in its own .text segment.

branches: 1.32.16;
Don't treat NUL (EOF) as SPACE. All the code that uses _PROP_ISSPACE() checks
explicitly for _PROP_EOF() anyway, and this can be abused to cause run beyond
the end of buffer DoS (Mateusz Kocielski)
XXX: pullup-7

branches: 1.31.12;
Bit-un-rot the proplib portability layer.

branches: 1.30.6;
Wipe out the last vestiges of POOL_INIT with one swift stroke. In
most cases, use a proper constructor. For proplib, give a local
equivalent of POOL_INIT for the kernel object implementation. This
way the code structure can be preserved, and a local link set is
not hazardous anyway (unless proplib is split to several modules,
but that'll be the day).

tested by booting a kernel in qemu and compile-testing i386/ALL

Get rid of locks with static initializers using once and atomic ops.
This makes proplib simplelock-free.

Fix two race conditions in proplib library. In prop_dictionary and prop_number
there was small window in which was entry left in rb tree with reference
count 0 which lead to rb tree coruption when another thread picked this up
before it was released.
Add 2 new members to the prop_object_t which are used for locking/unlocking
rb tree guard mutex.

Ok by joerg@, core@, thorpej@

branches: 1.27.4;
Fix an issue introduced with the prop_stack change: It's not a good idea to
return a 3-value enum from a function declared to return bool. This broke
the recurse case for prop_object_equals(). Instead, declare the object type
equals routine to return a _prop_object_equals_rv_t.

Give the same treatment to the object type free routines: declare them to
return a _prop_object_free_rv_t, and consistently check those return values
againt the enum type.

Tidy up some whitespace while we're here.

revert the recent "use trylock to hide locking errors" changes
and their associated changes, in particular, the following versions.
prop_array.c 1.13-1.16
prop_dictionary.c 1.25-1.28
prop_object_impl.h 1.21-1.25
ok'ed by core@.

branches: 1.25.2;
Change the argument of _PROP_RWLOCK_TRYRDLOCK() to the lock member and not
its address to be consistent with all the other macros.

Replace _prop_rwlock_tryrdlock() by a macro _PROP_RWLOCK_TRYRDLOCK().
This avoids build failures in stand alone binaries (e.g. bootloaders)
where there is no "pa_rwlock" member in the "_prop_array" structure.

Fix a "#define<tab><tab>" that crept in to previous.

prop_object_iterator_next() and prop_object_iterator_reset() were
acquiring the rwlock recursively in some paths.

Introduce _prop_rwlock_tryrdlock() and use it in these functions, so
that the rwlock is *always* acquired once, while here add some
_PROP_RWLOCK_OWNED asserts to verify.

This definitely fixes the "locking against myself" panics.

Do not acquire the rwlock recursively in prop_{array,dictionary}_iterator_reset()
when prop_{array,dictionary}_copyout_ioctl() is called.

Introduce _PROP_RWLOCK_OWNED() which is a KASSERT(rw_lock_held(lock))
and use it in those two functions, also acquire the rwlock in other
places where it is required now.

This fixes a LOCKDEBUG panic "locking against myself", as reported by
Geoff C. Wing in current-users@.

Remove clause 3 and 4 from TNF licenses

branches: 1.19.4;
Include sys/simplelock.h in prop_object_impl.h.

Avoid using unbound amount of stack frames in prop_object_equal
by using a dynamic stack as well. Reorder arguments for the internalizer
as the iteration is always present and should go before possibly
NULL arguments.

Reviewed by mjf@ and adrianp@

branches: 1.17.2;
libprop is currently using a recursive parser. While this is fine for
userland, deeply nested arrays and dictionaries can easily overflow
the kernel stack and thereby force a panic.

Fix the internalizer and prop_object_release to use a separate call
stack and alter the dictionary and array handling to not recurse on
the C stack. The default stack has an inline depth of 16 elements,
which should keep the overhead reasonable.

This issue was found by Pavel Cahyna and Jachym Holecek.

Additionally add a limit for prop_object_copyin_ioctl to prevent user
programs from temporary allocating unbound amount of kernel memory.
Allow malloc to fail so that tight loops of userland processes can't
force panics by exhausting the kernel map.

Tested with the sample exploit of Jachym, his test suite and reviewed
by himself (initial patch), Christos Zoulas and Jason Thorpe.

boolean_t -> bool
TRUE -> true
FALSE -> false

branches: 1.15.4;
lockmgr > rwlock

Consider '\r' to be white space. Discussed with and also kept as local
change by freza. With this change, DOS style line endings work.

prop_array_internalize and prop_dictionary_internalize are basically the
same code. Refactor it into _prop_generic_internalize, which gets passed
the second-level tag.

Pass an ipl argument to pool_init/POOL_INIT to be used when initializing
the pool's lock.

branches: 1.11.2; 1.11.4; 1.11.8;
Backout size_t casts - lint has been fixed.

Sprinkle a few size_t casts to avoid conversion warnings.

Add a _PROP_ARG_UNUSED that expands to __unused on NetBSD and to nothing
everywhere else. Use it where Christos decided to use __unused in this
code.

de-lint.

fix build issue for !_KERNEL
fixes pr lib/34708, credit Henning Petersen via t-online.de

- Add a reader-writer lock to array and dictionary objects that ensures that
the data structure is internally consistent in the face of multiple threads
accessing it concurrently. This is not designed to provide application-
level semantic consistency; applications are responsible for that locking
protocol should it be necessary.
- Rename _PROP_MUTEX_DECL() to _PROP_MUTEX_DECL_STATIC().

Adapt Matt Thomas's red-black tree implementation for use in proplib. This
has a significant code size savings over <sys/tree.h>.

Also change prop_number_t to store all number objects in an r-b tree,
only ever allocating one object for any given number (we can do this
because numbers are immutable). This results in significant run-time
memory savings.

Several changes to proplib:
- Arrays can now be externalized and internalized in the same way
dictionaries can.
- Add new "externalize to file" and "internalize from file" functions
to make reading a property list from a file and writing a property
list to a file more convenient.
- Many assertions in the object implementations are gone. Instead,
calling an accessor for one object type with a different object type
as an argument will return a suitable "invalid" value.
- prop_object_type() now returns a new PROP_TYPE_UNKNOWN value if called
with a NULL object.
- Externalized property lists now contain a reference to the Apple XML
plist DTD.
- Add a new prop_ingest(3) facility, which provides a convenient way to
translate a dictionary into an arbitrary binary representation.

branches: 1.3.2;
Move all of the locking compatibility definitions into prop_object_impl.h.

- Add prop_array_equals(), prop_dictionary_equals(), and
prop_dictionary_keysym_equals(), and prop_object_equals() functions.
- Use realloc() where it makes sense. There will be more changes in this
area.
- Add a _prop_object_type structure that is used internally to keep
information about the object types. Decreases the footprint of the
objects slightly by replacing several pointers with just one.

Initial commit of proplib, a library for manipulating property lists and
converting to/from an XML external representation (based on Apple XML
property lists). Works in the kernel and user space.

The local copy of rb_tree code has been unused for quite some time. So
we can just remove it, and adjust callers to use the "real" rbtree
function names.

Addresses PR lib/44090

Bit-un-rot the proplib portability layer.

branches: 1.9.2;
_prop_rb_tree_insert_node() now returns true/false to indicate if the
insertion succeeded. Update existing usage that arranges for insertions
to always succeed to assert that they do.

branches: 1.8.2;
Remove clause 3 and 4 from TNF licenses

branches: 1.7.8;
boolean_t -> bool
TRUE -> true
FALSE -> false

branches: 1.6.2; 1.6.4; 1.6.12;
Add a _PROP_ARG_UNUSED that expands to __unused on NetBSD and to nothing
everywhere else. Use it where Christos decided to use __unused in this
code.

de-lint.

Make it compile with -Wextra on big endian machines (at lest the #if in the
definition of struct rb_ndoe on the endianess is the only explanation I have
why nobody complained about this on i386 - I don't understand why it makes a
difference for gcc though)

Make this compile with -Wunused and handle the case where __predict_false()
is not defined (non-NetBSD environments).

Take the node indices out of the same namespace that <sys/tree.h> uses.

Adapt Matt Thomas's red-black tree implementation for use in proplib. This
has a significant code size savings over <sys/tree.h>.

Also change prop_number_t to store all number objects in an r-b tree,
only ever allocating one object for any given number (we can do this
because numbers are immutable). This results in significant run-time
memory savings.

The local copy of rb_tree code has been unused for quite some time. So
we can just remove it, and adjust callers to use the "real" rbtree
function names.

Addresses PR lib/44090

Bit-un-rot the proplib portability layer.

branches: 1.8.6;
Rename rb.h to rbtree.h, as it is more appropriate (c.f. ptree.h). Also
helps find code that hasn't been updated to use the new rbtree API.

Change tree op members/typedefs to rbto_compare_* from rb_compare_*