Home | History | Annotate | Download | only in ftp
History log of /src/usr.bin/ftp/ssl.c
RevisionDateAuthorComments
 1.20  25-Sep-2024  christos pass some lint.
 1.19  19-Jul-2024  lukem ftp: improve -b documentation

Order -b bufsize in the synopsis.
Document the actual default value.
 1.18  19-Feb-2024  christos default is now 16K
 1.17  18-Feb-2024  christos Add -b <buflen> to specify the buffer size.
 1.16  16-May-2023  christos Ignore EOF from remote.
 1.15  05-May-2023  lukem add timeout for ssl connect

Implement a timeout for SSL connection setup, using -q QUITTIME,
defaulting to 60 seconds.
SSL_connect(3) (unlike connect(2)) doesn't timeout by default.

Adapt ssl error messages destination: if unexpected error
from local API, use warn()/warnx() to stderr;
if expected error from a network operation (e.g., timeouts),
use fprintf to ttyout (which might be stdout).

Consistently use ftp_poll() instead of select();
ssl.c (using select()) was added 7 years after the
previous uses of select() were converted to poll().

Check EAGAIN as well as existing EINTR error from ftp_poll(),
for portability.
 1.14  09-Apr-2023  lukem Simplify includes

Include "ftp_var.h" instead of various system headers and "extern.h".
 1.13  25-Feb-2023  mlelstv Add option sslnoverify to control validation of SSL certificates.
Add netrc processing to fetch-mode (URL on command line) to enable options and autologin
via netrc.
Fix SSL cleanup in some error paths.

Certificate validation is now enabled by default. Set FTPSSLNOVERIFY=1 in environment
or configure a corresponding init macro via netrc to not validate certs (required if
you haven't installed a required CA certificate for OpenSSL).

Discussed with lukem@ on icb.
 1.12  12-Sep-2022  christos branches: 1.12.2;
Disable verification for now until we implement installation of trust anchors.
 1.11  30-Aug-2022  christos Add cert verification, together with an environment variable "NO_CERT_VERIFY",
to turn it off.
 1.10  03-Jun-2021  lukem use fetch_*() for I/O with SMALLPROG / !WITH_SSL builds

Adapt the SMALLPROG / -UWITH_SSL build to also use the fetch_*()
methods from ssl.c, instead of using stdio, as stdio isn't robust
when using interruptable signals.

Disable ssl-specific support in the fetch_*() methods if WITH_SSL
isn't defined, so SMALLPROG still doesn't have ssl support (as expected).

The resulting SMALLPROG binary is slightly larger than before
(e.g., 157KiB vs 153KiB on amd64).

Set version to 20210603 for this fix and the SO_KEEPALIVE fix for PR 56129.

PR install/56219
 1.9  06-Jan-2021  lukem branches: 1.9.4;
ftp: don't use restartable signals

Refactor to not rely upon restartable signals (SA_RESTART),
possibly fixing intermittent failures with -q QUITTIME.

ftp transfers: handle EINTR/EAGAIN in copy_bytes(),
instead of relying upon restartable signals.

http/https transfers: Explicitly print an error similar to
progressmeter() when timing-out for -Q QUITTIME in fetch_wait(),
and set errno to ETIMEDOUT so that the warn() in fetch_url()
prints a more accurate error message.

PR/55857
 1.8  07-Apr-2019  christos branches: 1.8.2;
redo the connection waiting handling to make it more clear.
 1.7  04-Apr-2019  christos Make fetch_read() return size_t like fread() does. It is bogus to
have one backing implementation that returns different values and
types than the other. Handle error setting properly; i.e. bail
out if the internal read returned an error. Now we get a proper
error message when the the server resets our connection instead of
a warning that the right failed with an invalid argument.

The server used for testing was:
http://capeweather.dyndns.org:8080/graphs/3474.png
Which seems to be unreliable :-)
 1.6  06-Feb-2018  christos branches: 1.6.4;
explicitly include <string.h> since OpenSSL-1.1 does not do it for us.
 1.5  16-Sep-2015  joerg branches: 1.5.8;
Workaround const issues of SSL_set_tlsext_host_name.
 1.4  12-Sep-2015  wiz servername cannot be NULL here.

Noted by joerg@.
 1.3  12-Sep-2015  wiz Add Server Name Indication (SNI) support for https.

Needed for e.g. some github URLs.
 1.2  24-Dec-2012  christos branches: 1.2.2; 1.2.6; 1.2.10; 1.2.14;
backwards compatible with netbsd-6.
 1.1  21-Dec-2012  christos PR/47276: Add https support
 1.2.14.2  13-Mar-2016  martin Pull up following revision(s) (requested by nonakap in ticket #1133):
usr.bin/ftp/fetch.c: revision 1.208-1.221
usr.bin/ftp/cmds.c: revision 1.136-1.137
usr.bin/ftp/ssl.c: revision 1.5
usr.bin/ftp/ftp.c: revision 1.165-1.166
usr.bin/ftp/ftp_var.h: revision 1.84

Workaround const issues of SSL_set_tlsext_host_name.

Use the proper format "[IPv6 address]:port" when reporting connection
attempts to IPv6 endpoints.

(Hopefully) fix build without IPv6 support

Try to factor out some code, this is completely out of control.

Separate no_proxy handling.

Factor the proxy handling code out.

Fix compile failure without WITH_SSL.

PR/50438: NONAKA Kimihiro: ftp(1): CONNECT method support

make DPRINTF/DWARN always statements.

Fix to connect https via proxy.

Fix ttyout message.

Simplify and factor out connect message

Split the position/size parsing into a separate function.

Mark function as only needed with ssl.

Fix downloads of local files using file:// URLs

Initialize the token match pointer.

use sizeof() and array notation.
CID 1354295: Array overrun.
 1.2.14.1  05-Nov-2015  riz Pull up following revision(s) (requested by wiz in ticket #981):
usr.bin/ftp/ftp.1: revision 1.135
usr.bin/ftp/ssl.c: revision 1.3
usr.bin/ftp/ssl.c: revision 1.4
usr.bin/ftp/ssl.h: revision 1.3
usr.bin/ftp/version.h: revision 1.86
usr.bin/ftp/version.h: revision 1.87
usr.bin/ftp/fetch.c: revision 1.207
usr.bin/ftp/main.c: revision 1.123
Add -x xferbufsize to set xferbuf size.
Implement -x xferbufsize set the socket send and receive buffer size,
as per 'xferbuf' in interactive mode.
Patch from Nicholas Mills (via private mail), with minor adjustment by me.
Add Server Name Indication (SNI) support for https.
Needed for e.g. some github URLs.
Bump version for SNI support.
servername cannot be NULL here.
Noted by joerg@.
 1.2.10.3  27-Aug-2016  bouyer Apply patch, requested by nonaka in ticket #1375:
src/usr.bin/ftp/cmds.c: patch
src/usr.bin/ftp/fetch.c: patch
src/usr.bin/ftp/ftp.1: patch
src/usr.bin/ftp/ftp.c: patch
src/usr.bin/ftp/ftp_var.h: patch
src/usr.bin/ftp/main.c: patch
src/usr.bin/ftp/ssl.c: patch
src/usr.bin/ftp/ssl.h: patch
src/usr.bin/ftp/version.h: patch
Update ftp(1) to version 20150912, adding https via proxy support.
 1.2.10.2  17-Dec-2013  bouyer Apply patch, requested by tron in ticket #997:
usr.bin/ftp/Makefile patch
usr.bin/ftp/cmds.c patch
usr.bin/ftp/cmdtab.c patch
usr.bin/ftp/extern.h patch
usr.bin/ftp/fetch.c patch
usr.bin/ftp/ftp.1 patch
usr.bin/ftp/ftp.c patch
usr.bin/ftp/ftp_var.h patch
usr.bin/ftp/main.c patch
usr.bin/ftp/progressbar.c patch
usr.bin/ftp/ssl.c patch
usr.bin/ftp/ssl.h patch
usr.bin/ftp/util.c patch
usr.bin/ftp/version.h patch

Add HTTPS support to ftp(1).
 1.2.10.1  24-Dec-2012  bouyer file ssl.c was added on branch netbsd-6 on 2013-12-17 21:07:59 +0000
 1.2.6.2  25-Feb-2013  tls resync with head
 1.2.6.1  24-Dec-2012  tls file ssl.c was added on branch tls-maxphys on 2013-02-25 00:30:35 +0000
 1.2.2.2  23-Jan-2013  yamt sync with head
 1.2.2.1  24-Dec-2012  yamt file ssl.c was added on branch yamt-pagecache on 2013-01-23 00:06:38 +0000
 1.5.8.3  12-Sep-2022  martin Catch up to current, requested by christos in ticket #1763:

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/cmds.c up to 1.141
usr.bin/ftp/complete.c up to 1.47
usr.bin/ftp/domacro.c up to 1.23
usr.bin/ftp/extern.h up to 1.82
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp.c up to 1.174
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/progressbar.c up to 1.24
usr.bin/ftp/progressbar.h up to 1.9
usr.bin/ftp/ssl.c up to 1.12
usr.bin/ftp/ssl.h up to 1.5
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

ftp(1): validate address from PASV and LPSV response.
ftp(1): use raw write(2) instead of fwrite(3) to avoid stream
corruption because of the progress bar interrupts.
Fixes for PR 56219 and PR 55857.
PR 57003: Support relative redirects.
 1.5.8.2  12-Sep-2022  martin Backout ticket #1763 for now - trust anchors are not solved.
 1.5.8.1  12-Sep-2022  martin Catch up to current, requested by christos in ticket #1763:

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/cmds.c up to 1.141
usr.bin/ftp/complete.c up to 1.47
usr.bin/ftp/domacro.c up to 1.23
usr.bin/ftp/extern.h up to 1.82
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp.c up to 1.174
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/progressbar.c up to 1.24
usr.bin/ftp/progressbar.h up to 1.9
usr.bin/ftp/ssl.c up to 1.11
usr.bin/ftp/ssl.h up to 1.5
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

ftp(1): validate address from PASV and LPSV response.
ftp(1): use raw write(2) instead of fwrite(3) to avoid stream
corruption because of the progress bar interrupts.
Fixes for PR 56219 and PR 55857.
PR 57003: Support relative redirects.
 1.6.4.1  10-Jun-2019  christos Sync with HEAD
 1.8.2.5  12-Sep-2022  martin Catch up to current, requested by christos in ticket #1523

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/ssl.c up to 1.12
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

PR 57003: Support relative redirects.
 1.8.2.4  12-Sep-2022  martin Back out ticket #1523 for now - trust anchor validation is not solved.
 1.8.2.3  12-Sep-2022  martin Catch up to current, requested by christos in ticket #1523

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/ssl.c up to 1.11
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

PR 57003: Support relative redirects.
 1.8.2.2  14-Jun-2021  martin Pull up following revision(s) (requested by lukem in ticket #1295):

usr.bin/ftp/ssl.c: revision 1.10
usr.bin/ftp/ssl.h: revision 1.5
usr.bin/ftp/version.h: revision 1.93
usr.bin/ftp/Makefile: revision 1.39

use fetch_*() for I/O with SMALLPROG / !WITH_SSL builds

Adapt the SMALLPROG / -UWITH_SSL build to also use the fetch_*()
methods from ssl.c, instead of using stdio, as stdio isn't robust
when using interruptable signals.

Disable ssl-specific support in the fetch_*() methods if WITH_SSL
isn't defined, so SMALLPROG still doesn't have ssl support (as expected).

The resulting SMALLPROG binary is slightly larger than before
(e.g., 157KiB vs 153KiB on amd64).

Set version to 20210603 for this fix and the SO_KEEPALIVE fix for PR 56129.

PR install/56219
 1.8.2.1  29-Jan-2021  martin Pull up following revision(s) (requested by lukem in ticket #1190):

usr.bin/ftp/progressbar.c: revision 1.24
usr.bin/ftp/ssl.c: revision 1.9
usr.bin/ftp/progressbar.h: revision 1.9
usr.bin/ftp/ftp.c: revision 1.171
usr.bin/ftp/version.h: revision 1.92

ftp: don't use restartable signals

Refactor to not rely upon restartable signals (SA_RESTART),
possibly fixing intermittent failures with -q QUITTIME.
ftp transfers: handle EINTR/EAGAIN in copy_bytes(),
instead of relying upon restartable signals.

http/https transfers: Explicitly print an error similar to
progressmeter() when timing-out for -Q QUITTIME in fetch_wait(),
and set errno to ETIMEDOUT so that the warn() in fetch_url()
prints a more accurate error message.

PR/55857
 1.9.4.1  06-Jun-2021  cjep sync with head
 1.12.2.5  13-Oct-2024  martin Pull up following revision(s) (requested by riastradh in ticket #970):

tests/usr.bin/Makefile: revision 1.42
usr.bin/ftp/ruserpass.c: revision 1.34
usr.bin/ftp/main.c: revision 1.130
usr.bin/ftp/ssl.c: revision 1.17
usr.bin/ftp/main.c: revision 1.131
usr.bin/ftp/ssl.c: revision 1.18
usr.bin/ftp/main.c: revision 1.132
usr.bin/ftp/ssl.c: revision 1.19
usr.bin/ftp/main.c: revision 1.133
distrib/sets/lists/tests/mi: revision 1.1342
usr.bin/ftp/ftp.1: revision 1.151
usr.bin/ftp/ftp.1: revision 1.152
usr.bin/ftp/progressbar.c: revision 1.25
usr.bin/ftp/ftp.1: revision 1.153
usr.bin/ftp/progressbar.c: revision 1.26
usr.bin/ftp/ftp.1: revision 1.155
usr.bin/ftp/ftp.1: revision 1.156
usr.bin/ftp/fetch.c: revision 1.239
usr.bin/ftp/ftp.1: revision 1.157
usr.bin/ftp/ftp.1: revision 1.158
usr.bin/ftp/ftp.1: revision 1.159
usr.bin/ftp/ftp_var.h: revision 1.87
etc/mtree/NetBSD.dist.tests: revision 1.208
usr.bin/ftp/ftp_var.h: revision 1.88
usr.bin/ftp/ftp_var.h: revision 1.89
usr.bin/ftp/cmds.c: revision 1.142
usr.bin/ftp/util.c: revision 1.168
usr.bin/ftp/cmds.c: revision 1.143
tests/usr.bin/ftp/custom_headers.sh: revision 1.1
usr.bin/ftp/ssl.c: revision 1.20
usr.bin/ftp/complete.c: revision 1.48
tests/usr.bin/ftp/Makefile: revision 1.1
tests/usr.bin/ftp/t_custom_headers.sh: revision 1.1
usr.bin/ftp/fetch.c: revision 1.240
usr.bin/ftp/fetch.c: revision 1.241
usr.bin/ftp/ftp.c: revision 1.176
usr.bin/ftp/ftp.c: revision 1.177
(all via patch)

ftp(1): wording and formatting improvements

Fix grammar issue with "Support values" reported in private mail.
Document all file transfer types in "type" and cross-reference that.
Consistency fixes in describing file transfer parameters and types.

Fix some mandoc -Tlint issues (except "useless macro: Tn").

Add -b <buflen> to specify the buffer size.

ftp: bump FTPBUFLEN from 4kB to 16kB
sourceforge.net returns a 5kB content-security-policy.
Analyzed by mlelstv@ who reports usual limits are between 4kB and 48kB.
default is now 16K

ftp: improve -b documentation

Order -b bufsize in the synopsis.

Document the actual default value.

ftp: improve units used in comments and errors
Use "KiB" instead of "K" in errors.
Clarify related comments.

pass some lint.

PR/58581: Sunil Nimmagadda: Add flag to allow specifying extra http header
fields.

ftp(1): Nix trailing whitespace in man page.
No functional change intended.

PR bin/58581: ftp(1) should allow specifying header fields in http requests
fix markup (h -> H), explain about multiple headers, fix usage (from RVP)

Don't forget the dot, use the intended macro name (I think),
and improve the wording a little. (All related to the -H option.)
ftp(1): Add test for custom HTTP header fields.

Based on a patch from Sunil Nimmagadda.

PR bin/58581: ftp(1) should allow specifying header fields in http
requests
 1.12.2.4  26-Aug-2024  martin Pull up following revision(s) (requested by wiz in ticket #814):

usr.bin/ftp/ssl.c: revision 1.16

Ignore EOF from remote.
 1.12.2.3  16-May-2023  martin Pull up following revision(s) (requested by lukem in ticket #171):

usr.bin/ftp/ssl.c: revision 1.15
usr.bin/ftp/util.c: revision 1.167
usr.bin/ftp/ftp.c: revision 1.175
usr.bin/ftp/version.h: revision 1.97

add timeout for ssl connect

Implement a timeout for SSL connection setup, using -q QUITTIME,
defaulting to 60 seconds.

SSL_connect(3) (unlike connect(2)) doesn't timeout by default.
Adapt ssl error messages destination: if unexpected error
from local API, use warn()/warnx() to stderr;
if expected error from a network operation (e.g., timeouts),
use fprintf to ttyout (which might be stdout).

Consistently use ftp_poll() instead of select();
ssl.c (using select()) was added 7 years after the
previous uses of select() were converted to poll().

Check EAGAIN as well as existing EINTR error from ftp_poll(),
for portability.
 1.12.2.2  16-May-2023  martin Pull up following revision(s) (requested by lukem in ticket #174):

usr.bin/ftp/ssl.c: revision 1.14

Simplify includes

Include "ftp_var.h" instead of various system headers and "extern.h".
 1.12.2.1  16-May-2023  martin Pull up following revision(s) (requested by lukem in ticket #170):

usr.bin/ftp/ssl.c: revision 1.13
usr.bin/ftp/fetch.c: revision 1.236
usr.bin/ftp/util.c: revision 1.166
usr.bin/ftp/main.c: revision 1.129
usr.bin/ftp/extern.h: revision 1.83
usr.bin/ftp/ftp.1: revision 1.148
usr.bin/ftp/cmdtab.c: revision 1.53
usr.bin/ftp/version.h: revision 1.96

Add option sslnoverify to control validation of SSL certificates.

Add netrc processing to fetch-mode (URL on command line) to enable options and autologin
via netrc.

Fix SSL cleanup in some error paths.

Certificate validation is now enabled by default. Set FTPSSLNOVERIFY=1 in environment
or configure a corresponding init macro via netrc to not validate certs (required if
you haven't installed a required CA certificate for OpenSSL).

Discussed with lukem@ on icb.

RSS XML Feed