cgdconfig(8): Add support for shared keys.

New clause `shared <id> algorithm <alg> subkey <info>' in a keygen
block enables `cgdconfig -C' to reuse a key between different params
files, so you can, e.g., use a single password for multiple disks.
This is better than simply caching the password itself because:

- Hashing the password is expensive, so it should only be done once.

Suppose your budget is time t before you get bored, and you
calibrate password hash parameters to unlock n disks before you get
bored waiting for `cgdconfig -C'.

. With n password hashings the adversary's cost goes up only by a
factor of t/n.
. With one password hashing and n subkeys the adversary's cost goes
up by a factor of n.

And if you ever add a disk, rehashing it will make `cgdconfig -C'
go over budget, whereas another subkey adds negligible cost to you.

- Subkeys work for other types of keygen blocks, like shell_cmd,
which could be used to get a key from a hardware token that needs a
button press.

The <info> parameter must be different for each params file;
everything else in the keygen block must be the same. With this
clause, the keygen block determines a shared key used only to derive
keys; the actual key used by cgdconfig is derived from the shared key
by the specified algorithm.

The only supported algorithm is hkdf-hmac-sha256, which uses
HKDF-Expand of RFC 5869 instantiated with SHA-256.

Example:

algorithm aes-cbc;
iv-method encblkno1;
keylength 128;
verify_method none;
keygen pkcs5_pbkdf2/sha1 {
iterations 39361;
salt AAAAgMoHiYonye6KogdYJAobCHE=;
shared "pw" algorithm hkdf-hmac-sha256
subkey AAAAgFlw0BMQ5gY+haYkZ6JC+yY=;
};

The key used for this disk will be derived by

HKDF-HMAC-SHA256_k(WXDQExDmBj6FpiRnokL7Jg==),

where k is the outcome of PBKDF2-SHA1 with the given parameters.

Note that <info> encodes a four-byte prefix giving the big-endian
length in bits of the info argument to HKDF, just like all other bit
strings in cgdconfig parameters files.

If you have multiple disks configured using the same keygen block
except for the info parameter, `cgdconfig -C' will only prompt once
for your passphrase, generate a shared key k with PBKDF2 as usual,
and then reuse it for each of the disks.

cgdconfig(8): New -t operation just prints the derived key in base64.

For testing purposes.

New cgd cipher adiantum.

Adiantum is a wide-block cipher, built out of AES, XChaCha12,
Poly1305, and NH, defined in

Paul Crowley and Eric Biggers, `Adiantum: length-preserving
encryption for entry-level processors', IACR Transactions on
Symmetric Cryptology 2018(4), pp. 39--61.

Adiantum provides better security than a narrow-block cipher with CBC
or XTS, because every bit of each sector affects every other bit,
whereas with CBC each block of plaintext only affects the following
blocks of ciphertext in the disk sector, and with XTS each block of
plaintext only affects its own block of ciphertext and nothing else.

Adiantum generally provides much better performance than
constant-time AES-CBC or AES-XTS software do without hardware
support, and performance comparable to or better than the
variable-time (i.e., leaky) AES-CBC and AES-XTS software we had
before. (Note: Adiantum also uses AES as a subroutine, but only once
per disk sector. It takes only a small fraction of the time spent by
Adiantum, so there's relatively little performance impact to using
constant-time AES software over using variable-time AES software for
it.)

Adiantum naturally scales to essentially arbitrary disk sector sizes;
sizes >=1024-bytes take the most advantage of Adiantum's design for
performance, so 4096-byte sectors would be a natural choice if we
taught cgd to change the disk sector size. (However, it's a
different cipher for each disk sector size, so it _must_ be a cgd
parameter.)

The paper presents a similar construction HPolyC. The salient
difference is that HPolyC uses Poly1305 directly, whereas Adiantum
uses Poly1395(NH(...)). NH is annoying because it requires a
1072-byte key, which means the test vectors are ginormous, and
changing keys is costly; HPolyC avoids these shortcomings by using
Poly1305 directly, but HPolyC is measurably slower, costing about
1.5x what Adiantum costs on 4096-byte sectors.

For the purposes of cgd, we will reuse each key for many messages,
and there will be very few keys in total (one per cgd volume) so --
except for the annoying verbosity of test vectors -- the tradeoff
weighs in the favour of Adiantum, especially if we teach cgd to do
>>512-byte sectors.

For now, everything that Adiantum needs beyond what's already in the
kernel is gathered into a single file, including NH, Poly1305, and
XChaCha12. We can split those out -- and reuse them, and provide MD
tuned implementations, and so on -- as needed; this is just a first
pass to get Adiantum implemented for experimentation.

Centralize the base rump libraries into a variable used by all the other
Makefiles so that we can make changes to it centrally as needed and have
less mess. Fixes the sun2 build that needs rumpvfs after librump after
the latest changes.

librump depends on vfs so add a dependency on for sun2.

branches: 1.8.10;
Remove MKCRYPTO option.

Originally, MKCRYPTO was introduced because the United States
classified cryptography as a munition and restricted its export. The
export controls were substantially relaxed fifteen years ago, and are
essentially irrelevant for software with published source code.

In the intervening time, nobody bothered to remove the option after
its motivation -- the US export restriction -- was eliminated. I'm
not aware of any other operating system that has a similar option; I
expect it is mainly out of apathy for churn that we still have it.
Today, cryptography is an essential part of modern computing -- you
can't use the internet responsibly without cryptography.

The position of the TNF board of directors is that TNF makes no
representation that MKCRYPTO=no satisfies any country's cryptography
regulations.

My personal position is that the availability of cryptography is a
basic human right; that any local laws restricting it to a privileged
few are fundamentally immoral; and that it is wrong for developers to
spend effort crippling cryptography to work around such laws.

As proposed on tech-crypto, tech-security, and tech-userlevel to no
objections:

https://mail-index.netbsd.org/tech-crypto/2017/05/06/msg000719.html
https://mail-index.netbsd.org/tech-security/2017/05/06/msg000928.html
https://mail-index.netbsd.org/tech-userlevel/2017/05/06/msg010547.html

P.S. Reviewing all the uses of MKCRYPTO in src revealed a lot of
*bad* crypto that was conditional on it, e.g. DES in telnet... That
should probably be removed too, but on the grounds that it is bad,
not on the grounds that it is (nominally) crypto.

Add 3des-cbc tests with 192 bits key.

Add blowfish-cbc tests for 128, 256 and 448 bits keys.

Only build t_cgd_aes if MKCRYPTO==yes and MKRUMP==yes.

Don't build t_cgd_aes if ${MKCRYPTO} == "no".

Add tests for not-yet-committed cgd algorithm AES-XTS.

The tests are marked as expected failures.

branches: 1.2.28;
Retire the old C helper in h_img2cgd since we can now write everything
with a shellscript in terms of rump.cgdconfig and dd.

Add rudimentary cgd tests. The tests use cgd to transform a
plaintext into into an encrypted image and back into plaintext by
doing rump I/O on /dev/cgd. There is one test to check that giving
the same password for both encryption and decryption produces the
same plaintext and another to check that giving a different passwords
does not produce the same plaintext.

This could be fairly easily extended to test all feature of cgd
(hint hint). For example, now cgd.conf is included in cvs, but
the only reason for that is that without further hacking cgdconfig
uses /dev/random quality random to generate the salt for a
pkcsetcetc_kdf2 cgconfig -g, and making an automated test block on
the entropy pool is just not good form. Details are everything.

cgdconfig(8): Add support for shared keys.

New clause `shared <id> algorithm <alg> subkey <info>' in a keygen
block enables `cgdconfig -C' to reuse a key between different params
files, so you can, e.g., use a single password for multiple disks.
This is better than simply caching the password itself because:

- Hashing the password is expensive, so it should only be done once.

Suppose your budget is time t before you get bored, and you
calibrate password hash parameters to unlock n disks before you get
bored waiting for `cgdconfig -C'.

. With n password hashings the adversary's cost goes up only by a
factor of t/n.
. With one password hashing and n subkeys the adversary's cost goes
up by a factor of n.

And if you ever add a disk, rehashing it will make `cgdconfig -C'
go over budget, whereas another subkey adds negligible cost to you.

- Subkeys work for other types of keygen blocks, like shell_cmd,
which could be used to get a key from a hardware token that needs a
button press.

The <info> parameter must be different for each params file;
everything else in the keygen block must be the same. With this
clause, the keygen block determines a shared key used only to derive
keys; the actual key used by cgdconfig is derived from the shared key
by the specified algorithm.

The only supported algorithm is hkdf-hmac-sha256, which uses
HKDF-Expand of RFC 5869 instantiated with SHA-256.

Example:

algorithm aes-cbc;
iv-method encblkno1;
keylength 128;
verify_method none;
keygen pkcs5_pbkdf2/sha1 {
iterations 39361;
salt AAAAgMoHiYonye6KogdYJAobCHE=;
shared "pw" algorithm hkdf-hmac-sha256
subkey AAAAgFlw0BMQ5gY+haYkZ6JC+yY=;
};

The key used for this disk will be derived by

HKDF-HMAC-SHA256_k(WXDQExDmBj6FpiRnokL7Jg==),

where k is the outcome of PBKDF2-SHA1 with the given parameters.

Note that <info> encodes a four-byte prefix giving the big-endian
length in bits of the info argument to HKDF, just like all other bit
strings in cgdconfig parameters files.

If you have multiple disks configured using the same keygen block
except for the info parameter, `cgdconfig -C' will only prompt once
for your passphrase, generate a shared key k with PBKDF2 as usual,
and then reuse it for each of the disks.

Retire the old C helper in h_img2cgd since we can now write everything
with a shellscript in terms of rump.cgdconfig and dd.

Unfortunately rump does not provide the same magic as MAKEDEV does
for native /dev and create an alias for disk devices w/o partition
latter pointing at the raw partition, so for rump based tests we
actually have to calculate the concrete device name.

Use an idiom suggested by kre for this which also works for ports that
have kern.rawpartition > 4.

Fix quoting (quotes really do not nest...) and remove a bunch of it
that is harmless, but also pointless (in sh, quotes do not make strings,
everything is a string, rather they hide characters which would have
some other meaning unquoted (like spaces) - quotes are not needed around
strings like "descr" so remove them...

Be more consistent with line wrap style, try to avoid wrapping in the
middle of a (sh) word where possible. Avoid \ use when it is not needed.

Un-KNF (C style) - sh has no declarations, there is no need to leave
blank lines at the head of a function to mark the end of the declarations.

This should be a NFC - but the quoting really was broken before, just
was probably harmless breakage.

PR bin/53999 from rudolf (eq.cz)

Fix cgdconfig to report verification failures with gpt and mbr
verification methods (and not treat them as silent hard errors).
This also causes the cgd to be unconfigured when one of those
verification methods fails.

Add ATF tests to check that bad verification is reported, and
does not leave the cgd configured.

Patches from the PR applied.

branches: 1.11.30;
Check for RUMP programs before using them.

branches: 1.10.4; 1.10.10;
Expand tests for unaligned writes to cgd. No more xfail.

PR kern/44515
PR kern/44964

Instead of doing 'atf_check ... sh -c foo', just do 'atf_check ... -x foo'.

Force cleanup parts to exit with a success status. Failures in cleanup
should not be allowed by atf-run (although they currently are ignored).

test case for PR kern/44515

convert tests from oldstyle dd rif/rof to newstyle dd | rump.dd

branches: 1.5.2;
make this work when rawpart != d

Substitute a surgical rump_server configuration for rump_allserver
now that it's possible. With warm fs cache, the startup time of
the former is 0.01s and the latter 0.1s. With cold caches it's
0.2s vs 2s.

Use proper cleanup.

XXX: the atf sh "compiler" should check for errors.

Retire the old C helper in h_img2cgd since we can now write everything
with a shellscript in terms of rump.cgdconfig and dd.

Add rudimentary cgd tests. The tests use cgd to transform a
plaintext into into an encrypted image and back into plaintext by
doing rump I/O on /dev/cgd. There is one test to check that giving
the same password for both encryption and decryption produces the
same plaintext and another to check that giving a different passwords
does not produce the same plaintext.

This could be fairly easily extended to test all feature of cgd
(hint hint). For example, now cgd.conf is included in cvs, but
the only reason for that is that without further hacking cgdconfig
uses /dev/random quality random to generate the salt for a
pkcsetcetc_kdf2 cgconfig -g, and making an automated test block on
the entropy pool is just not good form. Details are everything.

Plaintext buffers are used directly for write() operations to the raw device.
Align them to the needs of cgd(4).

Don't play with "../.." in includes for h_macros.h; deal with it centrally.
Minor fixes.

branches: 1.1.2;
Add 3des-cbc tests with 192 bits key.

clang can't handle __aligned on anonymous structure initializers.

Plaintext buffers are used directly for write() operations to the raw device.
Align them to the needs of cgd(4).

Missed a spot -- one more 32-bit sign-compare issue.

Fix sign-compare issue on 32-bit systems.

Built fine on amd64, where all unsigned values are representable in
ssize_t, but I didn't try building on i386, where they're not.

New cgd cipher adiantum.

Adiantum is a wide-block cipher, built out of AES, XChaCha12,
Poly1305, and NH, defined in

Paul Crowley and Eric Biggers, `Adiantum: length-preserving
encryption for entry-level processors', IACR Transactions on
Symmetric Cryptology 2018(4), pp. 39--61.

Adiantum provides better security than a narrow-block cipher with CBC
or XTS, because every bit of each sector affects every other bit,
whereas with CBC each block of plaintext only affects the following
blocks of ciphertext in the disk sector, and with XTS each block of
plaintext only affects its own block of ciphertext and nothing else.

Adiantum generally provides much better performance than
constant-time AES-CBC or AES-XTS software do without hardware
support, and performance comparable to or better than the
variable-time (i.e., leaky) AES-CBC and AES-XTS software we had
before. (Note: Adiantum also uses AES as a subroutine, but only once
per disk sector. It takes only a small fraction of the time spent by
Adiantum, so there's relatively little performance impact to using
constant-time AES software over using variable-time AES software for
it.)

Adiantum naturally scales to essentially arbitrary disk sector sizes;
sizes >=1024-bytes take the most advantage of Adiantum's design for
performance, so 4096-byte sectors would be a natural choice if we
taught cgd to change the disk sector size. (However, it's a
different cipher for each disk sector size, so it _must_ be a cgd
parameter.)

The paper presents a similar construction HPolyC. The salient
difference is that HPolyC uses Poly1305 directly, whereas Adiantum
uses Poly1395(NH(...)). NH is annoying because it requires a
1072-byte key, which means the test vectors are ginormous, and
changing keys is costly; HPolyC avoids these shortcomings by using
Poly1305 directly, but HPolyC is measurably slower, costing about
1.5x what Adiantum costs on 4096-byte sectors.

For the purposes of cgd, we will reuse each key for many messages,
and there will be very few keys in total (one per cgd volume) so --
except for the annoying verbosity of test vectors -- the tradeoff
weighs in the favour of Adiantum, especially if we teach cgd to do
>>512-byte sectors.

For now, everything that Adiantum needs beyond what's already in the
kernel is gathered into a single file, including NH, Poly1305, and
XChaCha12. We can split those out -- and reuse them, and provide MD
tuned implementations, and so on -- as needed; this is just a first
pass to get Adiantum implemented for experimentation.

Plaintext buffers are used directly for write() operations to the raw device.
Align them to the needs of cgd(4).

Gracefully skip test if not enough space in temporary directory.

branches: 1.6.14;
Don't play with "../.." in includes for h_macros.h; deal with it centrally.
Minor fixes.

branches: 1.5.2;
AES XTS unit tests should now pass.

Switch to CHECK_LIBC for writing.

Add aes-cbc tests.

Don't use mktemp.

Add tests for not-yet-committed cgd algorithm AES-XTS.

The tests are marked as expected failures.

Plaintext buffers are used directly for write() operations to the raw device.
Align them to the needs of cgd(4).

Don't play with "../.." in includes for h_macros.h; deal with it centrally.
Minor fixes.

branches: 1.1.2;
Add blowfish-cbc tests for 128, 256 and 448 bits keys.

When run from py-anita/amd64 this test fails with:

cgdconfig: getfsspecname failed: no match for `wd0e'

as the virtual machine has root on dk0, dk0 at wd0 and trying to
open wd0e fails.

This tests runs without a rump kernel and therefore should not
even try to open configured devices on the host. Replace the
disks "wd0e" and "ld1e" with non-existant disks "dska" and "dskb".

cgdconfig(8): Add support for shared keys.

New clause `shared <id> algorithm <alg> subkey <info>' in a keygen
block enables `cgdconfig -C' to reuse a key between different params
files, so you can, e.g., use a single password for multiple disks.
This is better than simply caching the password itself because:

- Hashing the password is expensive, so it should only be done once.

Suppose your budget is time t before you get bored, and you
calibrate password hash parameters to unlock n disks before you get
bored waiting for `cgdconfig -C'.

. With n password hashings the adversary's cost goes up only by a
factor of t/n.
. With one password hashing and n subkeys the adversary's cost goes
up by a factor of n.

And if you ever add a disk, rehashing it will make `cgdconfig -C'
go over budget, whereas another subkey adds negligible cost to you.

- Subkeys work for other types of keygen blocks, like shell_cmd,
which could be used to get a key from a hardware token that needs a
button press.

The <info> parameter must be different for each params file;
everything else in the keygen block must be the same. With this
clause, the keygen block determines a shared key used only to derive
keys; the actual key used by cgdconfig is derived from the shared key
by the specified algorithm.

The only supported algorithm is hkdf-hmac-sha256, which uses
HKDF-Expand of RFC 5869 instantiated with SHA-256.

Example:

algorithm aes-cbc;
iv-method encblkno1;
keylength 128;
verify_method none;
keygen pkcs5_pbkdf2/sha1 {
iterations 39361;
salt AAAAgMoHiYonye6KogdYJAobCHE=;
shared "pw" algorithm hkdf-hmac-sha256
subkey AAAAgFlw0BMQ5gY+haYkZ6JC+yY=;
};

The key used for this disk will be derived by

HKDF-HMAC-SHA256_k(WXDQExDmBj6FpiRnokL7Jg==),

where k is the outcome of PBKDF2-SHA1 with the given parameters.

Note that <info> encodes a four-byte prefix giving the big-endian
length in bits of the info argument to HKDF, just like all other bit
strings in cgdconfig parameters files.

If you have multiple disks configured using the same keygen block
except for the info parameter, `cgdconfig -C' will only prompt once
for your passphrase, generate a shared key k with PBKDF2 as usual,
and then reuse it for each of the disks.

cgdconfig(8): New -T operation prints all generated keys in cgd.conf.

For testing purposes.

cgdconfig(8): New -t operation just prints the derived key in base64.

For testing purposes.