Home | History | Annotate | only in /src/usr.bin/ftp
History log of /src/usr.bin/ftp
RevisionDateAuthorComments
 1.44 19-Oct-2025  riastradh openssl: Stop dragging libpthread into every libcrypto user.

With the new _NETBSD_PTHREAD_CREATE_WEAK, libcrypto can have a call
to pthread_create without linking against libpthread, so it is not
necessary to link all libcrypto users against libpthread too.

Applications that link against libcrypto but not libpthread can't use
threaded OpenSSL functionality, of course -- OSSL_set_max_threads
will always fail in such applications.

This reverts all the makefile churn that I found for unnecessary
libpthread linkage since the openssl 3.5 import.

PR lib/59685: libcrypto should not depend on libpthread
 1.43 24-Sep-2025  mrg m68* triggers a bad-maybe clobbered warning, avoid it.

ftp.c:779:23: error: variable 'rc' might be clobbered by 'longjmp' or 'vfork' [-Werror=clobbered]
779 | off_t rc;
| ^~
 1.42 20-Sep-2025  mrg sun2 and GCC 14 gets a maybe clobbered wrong here, i think.
 1.41 19-Jul-2025  christos crypto needs threads
 1.40 03-Jun-2023  lukem branches: 1.40.2;
bsd.own.mk: rename GCC_NO_* to CC_WNO_*

Rename compiler-warning-disable variables from
GCC_NO_warning
to
CC_WNO_warning
where warning is the full warning name as used by the compiler.

GCC_NO_IMPLICIT_FALLTHRU is CC_WNO_IMPLICIT_FALLTHROUGH

Using the convention CC_compilerflag, where compilerflag
is based on the full compiler flag name.
 1.39 03-Jun-2021  lukem use fetch_*() for I/O with SMALLPROG / !WITH_SSL builds

Adapt the SMALLPROG / -UWITH_SSL build to also use the fetch_*()
methods from ssl.c, instead of using stdio, as stdio isn't robust
when using interruptable signals.

Disable ssl-specific support in the fetch_*() methods if WITH_SSL
isn't defined, so SMALLPROG still doesn't have ssl support (as expected).

The resulting SMALLPROG binary is slightly larger than before
(e.g., 157KiB vs 153KiB on amd64).

Set version to 20210603 for this fix and the SO_KEEPALIVE fix for PR 56129.

PR install/56219
 1.38 06-Sep-2020  mrg branches: 1.38.4;
add support for new GCC 9 warnings that may be too much to fix
right now. new address-of-packed-member and format-overflow
warnings have new GCC_NO_ADDR_OF_PACKED_MEMBER amd
GCC_NO_FORMAT_OVERFLOW variables to remove these warnings.

apply to a bunch of the tree. mostly, these are real bugs that
should be fixed, but in many cases, only by removing the 'packed'
attribute from some structure that doesn't really need it. (i
looked at many different ones, and while perhaps 60-80% were
already properly aligned, it wasn't clear to me that the uses
were always coming from sane data vs network alignment, so it
doesn't seem safe to remove packed without careful research for
each affect struct.) clang already warned (and was not erroring)
for many of these cases, but gcc picked up dozens more.
 1.37 21-May-2017  riastradh branches: 1.37.2; 1.37.12;
Remove MKCRYPTO option.

Originally, MKCRYPTO was introduced because the United States
classified cryptography as a munition and restricted its export. The
export controls were substantially relaxed fifteen years ago, and are
essentially irrelevant for software with published source code.

In the intervening time, nobody bothered to remove the option after
its motivation -- the US export restriction -- was eliminated. I'm
not aware of any other operating system that has a similar option; I
expect it is mainly out of apathy for churn that we still have it.
Today, cryptography is an essential part of modern computing -- you
can't use the internet responsibly without cryptography.

The position of the TNF board of directors is that TNF makes no
representation that MKCRYPTO=no satisfies any country's cryptography
regulations.

My personal position is that the availability of cryptography is a
basic human right; that any local laws restricting it to a privileged
few are fundamentally immoral; and that it is wrong for developers to
spend effort crippling cryptography to work around such laws.

As proposed on tech-crypto, tech-security, and tech-userlevel to no
objections:

https://mail-index.netbsd.org/tech-crypto/2017/05/06/msg000719.html
https://mail-index.netbsd.org/tech-security/2017/05/06/msg000928.html
https://mail-index.netbsd.org/tech-userlevel/2017/05/06/msg010547.html

P.S. Reviewing all the uses of MKCRYPTO in src revealed a lot of
*bad* crypto that was conditional on it, e.g. DES in telnet... That
should probably be removed too, but on the grounds that it is bad,
not on the grounds that it is (nominally) crypto.
 1.36 21-Dec-2012  christos PR/47276: Add https support
 1.35 14-Aug-2011  christos branches: 1.35.2; 1.35.4; 1.35.8;
fix gcc-4.5 warnings
 1.34 22-Jun-2011  mrg apply some -Wno-error and/or -fno-strict-aliasing.


all of this should be looked at closer, but some of them are not
very trivial.
 1.33 03-Feb-2010  roy Userland now builds and uses terminfo instead of termcap.

OK: core@, jdc@
 1.32 28-May-2007  tls Add new Makefile knob, USE_FORT, which extends USE_SSP by turning on the
FORTIFY_SOURCE feature of libssp, thus checking the size of arguments to
various string and memory copy and set functions (as well as a few system
calls and other miscellany) where known at function entry. RedHat has
evidently built all "core system packages" with this option for some time.

This option should be used at the top of Makefiles (or Makefile.inc where
this is used for subdirectories) but after any setting of LIB.

This is only useful for userland code, and cannot be used in libc or in
any code which includes the libc internals, because it overrides certain
libc functions with macros. Some effort has been made to make USE_FORT=yes
work correctly for a full-system build by having the bsd.sys.mk logic
disable the feature where it should not be used (libc, libssp iteself,
the kernel) but no attempt has been made to build the entire system with
USE_FORT and doing so will doubtless expose numerous bugs and misfeatures.

Adjust the system build so that all programs and libraries that are setuid,
directly handle network data (including serial comm data), perform
authentication, or appear likely to have (or have a history of having)
data-driven bugs (e.g. file(1)) are built with USE_FORT=yes by default,
with the exception of libc, which cannot use USE_FORT and thus uses
only USE_SSP by default. Tested on i386 with no ill results; USE_FORT=no
per-directory or in a system build will disable if desired.
 1.31 29-Jun-2005  christos Add NO_USAGE and NO_DEBUG so that we can fit in the floppies again.
 1.30 11-Feb-2005  jmc branches: 1.30.2;
As dsl points out, sysinst uses the progress bar so put it back on SMALLPROG
 1.29 10-Feb-2005  jmc Add NO_PROGRESS for -DSMALL
 1.28 10-Jan-2005  lukem Only compile in IPv6 support if ${USE_INET6} != "no"

MKINET6 is for providing IPv6 infrastructure.
USE_INET6 is for compiling IPv6 support into the programs (needs MKINET6).
 1.27 06-Jun-2004  christos Save approximately 8K by not including http authentication, extended status
messages and help strings when the appropriate options are set.
 1.26 21-Jan-2003  jhawk Abstract out progress bar support into progressbar.[ch], for inclusion
in external programs (conditionalized on -DSTANDALONE_PROGRESS).

The following moved from util.c to progressbar.c:
alarmtimer(), progressmeter(), psummary(), ptransfer(),
xsignal(), xsignal_restart()

The following moved from extern.h and ftp_var.h to progressbar.h:
STALLTIME, verbose, fromatty, progress, quit_time, ttywidth
 1.25 30-Nov-2002  lukem tweaks for fparseln(3) move from libutil to libc:
- remove #include <util.h> if nothing else needed it
- remove LDFLAGS+=-lutil if nothing else needed it
 1.24 17-Oct-2002  lukem - if SMALLPROG is defined, add -DNO_EDITCOMPLETE -DNO_ABOUT,
and don't bother linking with -ledit -ltermcap
- if SMALLPROG is not defined or SMALLPROG_INET6 is defined, add -DINET6
 1.23 12-Dec-1999  lukem branches: 1.23.6;
add dependency on version.h
 1.22 01-Oct-1999  lukem simple dependancy so addition of global vars to ftp_var.h is detected for main.c
 1.21 26-Sep-1999  lukem * replace ifdefs against __SVR4 and __linux__ with DIRENT_MISSING_D_NAMLEN;
it's more portable and more obvious
* remove the mkgmtime() && HAVE_TIMEGM stuff:
a) why should netbsd have to define HAVE_TIMEGM to compile cleanly?
b) foreign compiles of ftp should just be linked with working
timegm function

a more portable version of this ftp client will be released as a 3rdparty
product; no use polluting our code with half-baked attempts...
 1.20 02-Jul-1999  itojun branches: 1.20.2;
add dual-stack (IPv4/v6) support. hope I broke no other part...
 1.19 16-Apr-1999  lukem revert previous (creation of /usr/bin/pftp link);
* $FTPMODE is the documented way in ftp(1) to force passive, active,
gate-ftp, or autodetect.
* AFAIK, we haven't shipped any previous release with pftp in /usr/bin
* no link was made for gate-ftp (and i don't think that makes sense in
/usr/bin either)
* even though the link isn't made, the support for pftp, gate-ftp, and
the `-p' flag should remain since people may depend on having their
own link (e.g, ~/bin/pftp -> /usr/bin/ftp) or aliases which use
these things. it doesn't hurt to leave argv[0] checking in ftp's main().
 1.18 15-Apr-1999  mcr added LINKS= to make /usr/bin/pftp
 1.17 18-Nov-1998  itohy Temporary fix of UTC -> time_t conversion:
1. changed to use timegm(3) on NetBSD and
2. supply alternative code for other Unix-like OSs
(NetBSD ftp shall be portable for some reasons :-).

This fix closes PR #6448.

NOTE: This should be fixed again if a portable UTC to time_t
conversion method is specified in some standards.
 1.16 22-Jul-1998  lukem * send 'Connection: close' in HTTP/1.1 headers, preventing the
remove server from using a persistent connection, which speeds
up such requests.
* support http 301 and 302 redirects
* rewrite guts of url_get() to use fparseln() et al instead of
read(s,&p,1)... enables each in the header to be parsed
as necessary
* rename login to ftp_login, to remove conflict with util.h::login
* cleanup verbose messages during http proxy requests
 1.15 18-Oct-1997  lukem branches: 1.15.2;
don't define WARNS=1 here
 1.14 14-Oct-1997  christos Change CFLAGS to CPPFLAGS
 1.13 18-Aug-1997  lukem bugs fixed:
* don't interpret '-' or '|' when a local filename is determined from
the remote name (i.e, in mget, and in get with only one argument).
This is implemented using an extra argument to recvrequest().
Fixes a major security hole.
* clean up memory leak when using globulize()
* clean up a couple of comments
* fix wording in TNF copyright

features added:
* support for TIS fwtk gate-ftp servers:
* read defaults from $FTPSERVER && $FTPSERVERPORT
* start in gate-ftp mode if invoked as 'gate-ftp'
* toggle or set with 'gate [host [port]]'
 1.12 20-Jul-1997  lukem * use RCSID() && COPYRIGHT() macros
* cleanup for WARNS=1 (including some ugly '(void)&var' bits wrapped in
#ifdef __GNUC__ to shut up gcc warnings WRT setjmp/longjmp)
* use strtol() instead of atoi(), and more extensively check result of
conversion
* use u_int16_t instead of short or int for TCP port addresses
 1.11 24-Mar-1997  christos - Makefile cleanups
 1.10 14-Mar-1997  christos SMALLFTP->SMALL; remove SMALL from Makefile
 1.9 13-Mar-1997  christos Deal with SMALLFTP a bit differently [by compiling complete.c]
 1.8 19-Jan-1997  lukem New features:
* Command line editing via editline(3) library.
* Context sensitive command and file completion, including remote files.

Enhancements to auto-fetch feature:
* Support for http:// URLs using the http protocol, including proxy HTTP
support via $htty_proxy if it's defined.
* The connection is kept open between successive files on the same host.
(obviously, this does not count for http requests.)
* Return value of ftp is 0 on no error, or the offset in argv[] of the file
which failed (i.e., argv[x] failed, ftp returns x).
* If the path in an ftp URL or classic format line has a trailing '/',
cd to the path and enter interactive mode. Fixes [bin/3011], albiet
requiring the user to help ftp in determining the operation.

Other changes:
* '-P port' works for normal ftp, and is the default for all classic style
auto-fetch transfers and for ftp URLs that don't specify the port.
(previously it would just work for the first xfer.)
* Some code moved into separated files along logical divisions.
* Editing and completion can be compiled out with -DSMALLFTP.
 1.7 09-Jan-1997  tls RCS ID police
 1.6 22-Nov-1995  cgd add -Dunix to CFLAGS
 1.5 08-Sep-1995  tls Sync with 4.4lite2
 1.4 29-Aug-1994  mycroft branches: 1.4.2;
Add RCS ids.
 1.3 25-Aug-1994  cgd clean up import, no local changes.
 1.2 31-Jul-1993  mycroft Add RCS indentifiers.
 1.1 21-Mar-1993  cgd branches: 1.1.1; 1.1.2;
Initial revision
 1.1.2.1 25-Aug-1994  cgd import from 4.4-Lite
 1.1.1.2 05-Sep-1995  tls imported from 44lite2
 1.1.1.1 21-Mar-1993  cgd initial import of 386bsd-0.1 sources
 1.4.2.2 29-Aug-1994  mycroft Add RCS ids.
 1.4.2.1 29-Aug-1994  mycroft file Makefile was added on branch netbsd-1-0 on 1994-08-29 03:09:06 +0000
 1.15.2.2 23-Nov-1998  cgd pull up rev 1.17 from trunk (itohy)
 1.15.2.1 10-Nov-1998  cgd pull up rev(s) 1.16 from trunk and patch to work on 1.3 branch. (feyrer)
 1.20.2.1 27-Dec-1999  wrstuden Pull up to last week's -current.
 1.23.6.1 10-Feb-2003  jmc Pullup rev 1.26 (requested by grant in ticket #1115)
Separate progress reporting into a utility set of routines. Use
these to provide a standalone progress binary and integrate support
for this into sysinst.
 1.30.2.1 24-Jul-2005  tron Pull up revision 1.31 (requested by lukem in ticket #606):
Add NO_USAGE and NO_DEBUG so that we can fit in the floppies again.
 1.35.8.1 25-Feb-2013  tls resync with head
 1.35.4.1 17-Dec-2013  bouyer Apply patch, requested by tron in ticket #997:
usr.bin/ftp/Makefile patch
usr.bin/ftp/cmds.c patch
usr.bin/ftp/cmdtab.c patch
usr.bin/ftp/extern.h patch
usr.bin/ftp/fetch.c patch
usr.bin/ftp/ftp.1 patch
usr.bin/ftp/ftp.c patch
usr.bin/ftp/ftp_var.h patch
usr.bin/ftp/main.c patch
usr.bin/ftp/progressbar.c patch
usr.bin/ftp/ssl.c patch
usr.bin/ftp/ssl.h patch
usr.bin/ftp/util.c patch
usr.bin/ftp/version.h patch

Add HTTPS support to ftp(1).
 1.35.2.1 23-Jan-2013  yamt sync with head
 1.37.12.4 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1523

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/ssl.c up to 1.12
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

PR 57003: Support relative redirects.
 1.37.12.3 12-Sep-2022  martin Back out ticket #1523 for now - trust anchor validation is not solved.
 1.37.12.2 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1523

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/ssl.c up to 1.11
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

PR 57003: Support relative redirects.
 1.37.12.1 14-Jun-2021  martin Pull up following revision(s) (requested by lukem in ticket #1295):

usr.bin/ftp/ssl.c: revision 1.10
usr.bin/ftp/ssl.h: revision 1.5
usr.bin/ftp/version.h: revision 1.93
usr.bin/ftp/Makefile: revision 1.39

use fetch_*() for I/O with SMALLPROG / !WITH_SSL builds

Adapt the SMALLPROG / -UWITH_SSL build to also use the fetch_*()
methods from ssl.c, instead of using stdio, as stdio isn't robust
when using interruptable signals.

Disable ssl-specific support in the fetch_*() methods if WITH_SSL
isn't defined, so SMALLPROG still doesn't have ssl support (as expected).

The resulting SMALLPROG binary is slightly larger than before
(e.g., 157KiB vs 153KiB on amd64).

Set version to 20210603 for this fix and the SO_KEEPALIVE fix for PR 56129.

PR install/56219
 1.37.2.3 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1763:

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/cmds.c up to 1.141
usr.bin/ftp/complete.c up to 1.47
usr.bin/ftp/domacro.c up to 1.23
usr.bin/ftp/extern.h up to 1.82
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp.c up to 1.174
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/progressbar.c up to 1.24
usr.bin/ftp/progressbar.h up to 1.9
usr.bin/ftp/ssl.c up to 1.12
usr.bin/ftp/ssl.h up to 1.5
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

ftp(1): validate address from PASV and LPSV response.
ftp(1): use raw write(2) instead of fwrite(3) to avoid stream
corruption because of the progress bar interrupts.
Fixes for PR 56219 and PR 55857.
PR 57003: Support relative redirects.
 1.37.2.2 12-Sep-2022  martin Backout ticket #1763 for now - trust anchors are not solved.
 1.37.2.1 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1763:

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/cmds.c up to 1.141
usr.bin/ftp/complete.c up to 1.47
usr.bin/ftp/domacro.c up to 1.23
usr.bin/ftp/extern.h up to 1.82
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp.c up to 1.174
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/progressbar.c up to 1.24
usr.bin/ftp/progressbar.h up to 1.9
usr.bin/ftp/ssl.c up to 1.11
usr.bin/ftp/ssl.h up to 1.5
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

ftp(1): validate address from PASV and LPSV response.
ftp(1): use raw write(2) instead of fwrite(3) to avoid stream
corruption because of the progress bar interrupts.
Fixes for PR 56219 and PR 55857.
PR 57003: Support relative redirects.
 1.38.4.1 06-Jun-2021  cjep sync with head
 1.40.2.1 02-Aug-2025  perseant Sync with HEAD
 1.144 04-Oct-2024  christos Check bounds when copying to destination.
Remove const where the const string ended up being overwritten.
 1.143 25-Sep-2024  christos pass some lint.
 1.142 19-Jul-2024  lukem ftp: improve units used in comments and errors

Use "KiB" instead of "K" in errors.
Clarify related comments.
 1.141 06-Jan-2021  lukem branches: 1.141.6; 1.141.8;
ftp(1): fix description of "debug"

"debug" command and documentation got accidentally renamed
to "ftp_debug" 13 years ago, and was only partially fixed.
 1.140 06-Feb-2019  martin branches: 1.140.2;
Try to avoid a (bogus) fatal warning from clang.
 1.139 04-Feb-2019  mrg - add justquit() that always exits. use it to avoid unreachable code.
 1.138 20-Nov-2017  kre branches: 1.138.4;
Issue PWD commands to the server only when we actually
need the results, not speculatively, just in case we might.

Allows operation with some broken servers that get confused
by PWD commands in some situations, and saves server round
trips in the (modern) common case of
ftp ftp://path/name
where we never need to know the results from PWD.
 1.137 27-Feb-2016  christos branches: 1.137.8;
CID 1354295: Array overrun.
 1.136 06-Feb-2016  christos use sizeof() and array notation.
 1.135 22-Dec-2012  christos branches: 1.135.8;
document https support, mentioning certificates are not validated, and obey
https_proxy.
 1.134 15-Jan-2012  christos branches: 1.134.2; 1.134.6;
fix previous; use append instead of write.
 1.133 15-Jan-2012  christos make -R work the same in ftp:// like it does for http:// (i.e. work if
there is no local file).
 1.132 16-Sep-2011  joerg branches: 1.132.2;
Use __dead
 1.131 12-Jan-2010  lukem rename argument from "bool" to "val"
 1.130 13-Jul-2009  roy Rename internal getline() function to get_line() so it does
conflict with the soon to be added getline(3) libc function.
 1.129 12-Apr-2009  lukem Fix numerous WARNS=4 issues (-Wcast-qual -Wsign-compare).
 1.128 12-Apr-2009  lukem fix -Wshadow issues
 1.127 05-Dec-2008  lukem branches: 1.127.2;
correct a comment
 1.126 30-Sep-2008  lukem update copyrights
 1.125 10-May-2008  skd Add epsv6 and epsv to disable extended passive mode for ipv6 or both ipv4 and ipv6 respectively. This hack is due to our friends a Juniper Networks who break
epsv in ipv6. Should be fixed in ScreenOS 6.2.X.
 1.124 28-Apr-2008  martin branches: 1.124.2;
Remove clause 3 and 4 from TNF licenses
 1.123 24-May-2007  lukem branches: 1.123.10;
Display times in RFC2822 form rather than using ctime(3), since
the former is more explicit about the timezone offset.
 1.122 10-May-2007  lukem Use getline() instead of fparseln()
 1.121 18-Apr-2007  lukem Clean up use of confirm() and mbort() so that the current operation
is always passed in (instead of depending upon the 'mname' global).
For confirm(), if the second argument is NULL print the "Continue with <cmd>"
prompt. This fixes up the the display of interrupted prompts.
 1.120 17-Apr-2007  lukem * Implement -s srcaddr; uses srcaddr as the local IP address for all
connections.
Based on code in the version of ftp that FreeBSD had before they
replaced it with lukemftp.
* Move error message handling into ftp_connect() rather than in the
caller, so that more specific error reporting can occur.
* Improve consistency of various warning and error messages.
 1.119 11-Apr-2007  lukem getpass() can return NULL upon error in some implementations
(as documented in older standards documents, before the API was obsoleted).
Problem observed in tnftp on Solaris by Emil Mikulic.
 1.118 31-Jan-2006  christos rename debug to ftp_debug. grr libssh.
 1.117 31-Jan-2006  christos Rename xfoo() to ftp_foo() to avoid collisions with libssh. Don't ask.
 1.116 29-Jun-2005  christos Add NO_USAGE and NO_DEBUG so that we can fit in the floppies again.
 1.115 10-Jun-2005  lukem Implement:
int getline(FILE *stream, char *buf, size_t buflen, const char **errormsg)
Read a line from the FILE stream into buf/buflen using fgets(), so up
to buflen-1 chars will be read and the result will be NUL terminated.
If the line has a trailing newline it will be removed.
If the line is too long, excess characters will be read until
newline/EOF/error.
Various -ve return values indicate different errors, and errormsg
will be changed to an error description if it's not NULL.

Convert to use getline() instead of fgets() whenever reading user input
to ensure that an overly long input line doesn't leave excess characters
for the next input operation to accidentally use as input.

Zero out the password & account after we've finished with it.

Consistently use getpass(3) (i.e, character echo suppressed) when
reading the account data. For some reason, historically the "login"
code suppressed echo for Account: yet the "user" command did not!

Display the hostname in the "getaddrinfo failed" warning.

Appease some -Wcast-qual warnings. Fixing all of these requires
significant code refactoring. (mmm, legacy code).
 1.114 19-May-2005  lukem Use size_t instead of int where appropriate.
 1.113 19-May-2005  lukem Some const cleanups.
 1.112 11-Apr-2005  lukem gratuitous whitespace cleanup (before someone else jumps the gun...)
 1.111 11-Feb-2005  simonb branches: 1.111.2;
White space nit- don't put a space before/after increment/decrement
operators.
 1.110 09-Feb-2005  christos Factor out common string processing code eliminating static buffers,
making functions that should be static be static, and cleaning up
const usage. Added a guard against buffer overflow, but the domap function
is a bit too complicated for me to tackle right now. I will leave it
to the author; hi luke!
 1.109 03-Jan-2005  lukem Forbid filenames returned from mget that aren't in (or below) the
current directory.
The previous behaviour (of trusting the remote server's response when
retrieving the list of files to mget with prompting disabled) has been
in ftp ~forever, and has been a "known issue" for a long time.
Recently an advisory was published by D.J. Bernstein on behalf of
Yosef Klein warning of the problems with the previous behaviour, so
to alleviate concern I've fixed this with a sledgehammer.

Remember the local cwd after any operation which may change it.
Use "remotecwd" instead of "remotepwd".
 1.108 30-Oct-2004  dsl Add (unsigned char) cast to ctype functions
 1.107 20-Jul-2004  lukem Ensure that "mname" is set in ls() and mls() so that an aborted confirm()
prints the correct name.
Problem highlighted & suggested fix from PR [bin/17766] by Steve McClellan.
 1.106 20-Jul-2004  lukem If an ftp auto-fetch transfer is interrupted by SIGINT (usually ^C),
exit with 130 instead of 1 (or rarely, 0).
This allows an ftp auto-fetch in a shell loop to correctly terminate the loop.
Should fix PR [pkg/26351], and possibly others.
 1.105 06-Jun-2004  kleink Fix an inversed test for NO_STATUS in rev. 1.103; fixes the real issue
behind PR bin/25842.
 1.104 06-Jun-2004  pooka make this compile without NO_STATUS

fixes bin/25842 by Jukka Salmi
 1.103 06-Jun-2004  christos Save approximately 8K by not including http authentication, extended status
messages and help strings when the appropriate options are set.
 1.102 07-Aug-2003  agc Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22365, verified by myself.
 1.101 31-Jul-2003  lukem Invalidate remote directory completion cache if any command which
may change the remote contents completes successfully, including:
del, mdel, ren, mkdir, rmdir, quote, and all upload commands
Patch from Yar Tikhiy <yar@comp.chem.msu.su>.
 1.100 30-Nov-2002  lukem tweaks for fparseln(3) move from libutil to libc:
- remove #include <util.h> if nothing else needed it
- remove LDFLAGS+=-lutil if nothing else needed it
 1.99 12-Jul-2002  itojun strdup -> xstrdup (handle malloc error right)
 1.98 05-Jun-2002  lukem - when showing the final progress bar, replace "00:00 ETA" with the
elapsed time. (suggested by simonb)
- actually display transfer stats after a URL fetch. (bug introduced a
*long* time ago)
- update copyright & version
 1.97 07-May-2002  lukem Use "r+" instead of "r+w", since the latter is not standard.
Noted by <Steve.McClellan@radisys.com> in private email.
 1.96 06-May-2002  lukem - Only send port number in HTTP/1.1 Host: request if port != 80.
Fixes [bin/15415] from Takahiro Kambe <taca@sky.yamashina.kyoto.jp>
- Fix bad mode passed by mls() to recvrequest().
Fixes [bin/16642] from <steve.mcclellan@radisys.com>
 1.95 26-Dec-2001  lukem update copyrights
 1.94 19-Feb-2001  lukem minor knf
 1.93 15-Dec-2000  lukem invoke cmdtab.c_handler()s with argv[0] == c_name instead of the
supplied name. that way the full (unambiguous) name is displayed in
error messages and usage strings.
 1.92 15-Nov-2000  lukem - implement "mreget"; as per "mget" but uses "reget" instead of "get"
- add -N netrc and $NETRC, as methods to select an alternative .netrc file
- cache local user name and home directory for further use
- in mget(), use docase() instead of a local version to do the case
conversion.
 1.91 11-Oct-2000  is More format string cleanup by sommerfeld.
 1.90 01-Aug-2000  lukem - rename NO_QUAD to NO_LONG_LONG, QUAD* -> LL* and add ULL* (unsigned)
equivalents. name change suggested by Klaus Klein <kjk@netbsd.org>
- change defined(BSD4_4) || HAVE_SIN_LEN tests into HAVE_SOCKADDR_SA_LEN,
and set the latter if BSD4_4 exists
 1.89 30-Jul-2000  lukem clean up NO_QUAD support: create helper #defines and use as appropriate:
#define NOQUAD ! NOQUAD
------- ------ - ------
QUADF "%ld" "%lld"
QUADFP(x) "%" x "ld" "%" x "lld"
QUADT long long long
STRTOL(x,y,z) strtol(x,y,z) strtoll(x,y,z)
 1.88 18-Jul-2000  lukem add support for FEAT and OPTS commands with `features' and `opts'.
(from RFC 2389).

add support for MLST & MLSD (machine parseble listings) with 'mlst', 'mlsd'
and 'pmlsd' (mlsd |$PAGER) commands. (from draft-ietf-ftpext-mlst-11)

rename remotesyst() to getremoteinfo(), and modify to parse the result from
FEAT (if supported), and take into account the support for the various
extensions such as MDTM, SIZE, REST (STREAM), MLSD, and FEAT/OPTS.
put each feature into one of the following categories:
- known to work (explicit FEAT)
- unknown but assume works until explicit failure, when it's
then tagged as `known not to work'.
- known not to work (FEAT succeeded but didn't return anything,
or was unknown and then explicit failure)
assign results into features[] matrix.

add support to getreply() so that an optional callback will be called
for each line received from the server except for the first and last.
this is used in FEAT (and MLST) parsing.

modify various commands to check if REST (STREAM), MDTM and SIZE are
explicitly or implicitly supported before using.

fix `syst' when verbose is off.

minor knf (indent goto labels by one space, etc).

simply various command usage handlers by assuming that argv != NULL except
for quit() and disconnect().
 1.87 15-Jun-2000  lukem branches: 1.87.2;
* migrate the SYST parsing from setpeer() into a separate remotesyst().
call remotesyst() only when login has been successful
some servers don't let you run SYST until you've successfully logged in.
* in fetch_ftp(), always call setpeer() with autologin disabled, and use
the following ftp_login() to DTRT.
this prevents ftp from trying to login a second time if the first autologin
fails when connecting to a remote site anonymously using autofetch.
* reset unix_proxy and unix_server in cleanuppeer()
* missed a function conversion in the KNF sweep...
 1.86 28-May-2000  lukem Change `ls' to use the `LIST' and not `NLST' FTP protocol command.
Now that after many years on not caring we find certain popular
ftp servers are starting to obey RFC959 to the letter of the law
and will only return a list of filenames (not directories or
other filetypes) in the output of `NLST', then `LIST' is more useful
in this case. (Note that the aforementioned pedanticness means that
filename completion isn't as useful as it could be...)
Fixes [bin/8937] by David A. Gatwood <dgatwood@deepspace.mklinux.org>
 1.85 01-May-2000  lukem branches: 1.85.2;
convert to ANSI KNF
 1.84 01-May-2000  lukem * Add support for `fget localfile', which reads a list of filenames to
retrieve from localfile. Based on work by Darren Reed.
* Crank version.
* Update copyright dates.
 1.83 13-Apr-2000  lukem s/strtoq/strtoll/ (the latter is standardised)
 1.82 31-Jan-2000  lukem define private type `sigfunc' as
typedef void (*sigfunc) __P((int));
and replace use of sig_t and void (*)(int).

certain other OSes define sig_t differently to that (they add extra arguments),
and it causes problems due to function mismatches, etc...
 1.81 05-Dec-1999  lukem move version into separate file to reduce recompilation after version crank.
 1.80 26-Nov-1999  lukem * complete_remote(): use remglob("", ...) instead of remglob(".", ...),
for listings of the current working directory; some ftp servers don't
like `NLST .'.
[noted by Giles Lean <giles@nemeton.com.au>]
* recvrequest(): treat remote=="" as remote==NULL when calling command().
(to support the above change)
* support `[user@]' in `[user@]host' and `[user@]host[:][path]'.
[based on idea (and initial code) from David Maxwell <david@fundy.ca>]
* `idle' may be invoked without any args
* reformat some comments
* reformat usage string in program and man page
* call updateremotepwd() after successful login, not after successful connect
* always call setsockopt(, IPPROTO_IP, IP_TOS, ) (et al); using #if
defined(IPPROTO_IP) doesn't work on certain foreign systems where
enums instead of #defines are used...
[noted by Matthias Pfaller <leo@dachau.marco.de>]
 1.79 11-Nov-1999  lukem - implement updateremotepwd(); update the global variable `remotepwd'
to contain the remote working directory.
- add `set prompt', a user configurable prompt. (defaults to `ftp> ').
the following escape characters a la tcsh(1) are supported: %/, %m,
%M, and %n.
- add global var `username'; used by prompt code
- fix a couple of minor memory leaks
- bump version
 1.78 09-Nov-1999  lukem - split the version string into product and version
- be consistent about reporting the version between:
+ status command
+ about:version URL fetch
+ User-agent sent in http requests
 1.77 03-Nov-1999  lukem support `about:version'. also display the version in the output of `status'.
 1.76 24-Oct-1999  lukem new features:
- add `usage'; displays the usage of a command.
implemented by calling the c_handler() with argc = 0, argv = "funcname".
- add `passive auto'; does the same as $FTPMODE=auto.
- add `set [option value]'; display all options, or set an option to a value.
- add `unset option'; unset an option.
- add getoptionvalue() to retrieve an option's value, and replace a few
global variables with calls to this.
- implement cleanuppeer(), which resets various bits of state back to
`disconnected'. call in disconnect() and lostpeer().
- support completing on `options'.
- improve recovery after a SIGINT may have closed the connection.
XXX: there's still a couple to fix

other stuff:
- various consistency fixes in the man page.
- ensure that the command usage strings in the code and man page match reality.
- mput/mget: check that the connection still exists before each xfer.
- minor cosmetic changes in confirm().
- set code correctly in sizecmd() and modtime()
- don't need \n in err() strings.
- change lostpeer to take an argument (rather than casting (sig_t)lostpeer
in signal handlers)
- knf and whitespace police.
 1.75 13-Oct-1999  lukem * fix up confirm() (broke `a' and `p' in last commit)
* simplify main loop (don't need `top' variable any more)
* use a struct sockaddr_in6.sin6_addr for the result from inet_pton(),
rather than u_char buf[16]
* add a few more comments
 1.74 12-Oct-1999  lukem a few user interface and cosmetic tweaks:
* confirm(): move from util.c to cmds.c. display mnemonic string in its prompt.
add support for `q' (terminate current xfer), `?' (show help list)
* in various signal handlers, output a linefeed only if fromatty.
* if fgets(stdin) returned NULL (i.e, EOF), clearerr(stdin) because you don't
want future fgets to fail. this is not done for the fgets() in the main
command loop, since ftp will quit at that point.
* unless ftp is invoked with -a, don't retain the anonftp setting between
hosts (`ftp somehost:' sets anonftp, but you don't want that to `stick'
if you close that connection and open a new one).
 1.73 10-Oct-1999  lukem use sigjmp_buf for sigsetjmp(), instead of jmp_buf.
noted by Havard.Eidnes@runit.sintef.no.
 1.72 09-Oct-1999  lukem * use sigsetjmp()/siglongjump() instead of setjmp()/longjmp(); the latter
don't save the signal mask on some foreign systems.
* ensure signal handlers don't use stdio and do reset errno if they
don't exit with siglongjmp()
* use a common SIGINT handler for {send,recv}request()
 1.71 05-Oct-1999  lukem * factor out SIGINFO setting into a handler that is always active (but only
prints out info if bytes > 0). only set the handler if SIGINFO is defined
* hijack SIGQUIT to be the same as SIGINFO (foreign ports have this, and it's
annoying to have SIGQUIT dump core on netbsd when it prints info on other
systems)
* in {recv,send}request(), factor a lot of duplicated code out into a
`cleanup' section at the end
* rework shell() a bit
 1.70 05-Oct-1999  lukem add TNFi copyright to all files i've done more than a minor amount of work to...
 1.69 05-Oct-1999  lukem enhancments from Marc Horowitz <marc@mit.edu> to improve connection timeouts:
* implement xsignal_restart(), which only sets the SA_RESTART flag if
specifically requested
* xsignal() is now a wrapper to xsignal_restart(). INFO, USR1, USR2 and WINCH
are restartable, ALRM, INT, PIPE and QUIT are not.
* improve getreply()'s timeout code to take advantage of the above.

other changes:
* improve wording of how globbing works for `classic' URLs (host:path).
suggested by John Refling <johnr@imageworks.com> in relation to PRs
[bin/8519] and [bin/8520]
* always compile in the `edit' command even if NO_EDITCOMPLETE defined.
it's just a no-op in the latter case, which is more consistent to
the users.
* always compile in about: support (i.e, remove NO_ABOUT).
i'm entitled to some vanity in this program...
* clean up some whitespace
 1.68 01-Oct-1999  lukem If EPSV or EPRT fails, disable epsv4 for the rest of the current connection.
the disabled state can be overridden by toggling epsv4.

(I got sick of the errors about EPSV not being supported on almost
every server I connect to. This way we retain support for epsv4, but
it's not so whiny after the first failure...)
 1.67 01-Oct-1999  lukem * don't bother with "pathnames.h"; it contained just a single #define.
instead, include <paths.h> in the files that need it
* slightly reorg ftp_var.h
 1.66 01-Oct-1999  lukem * newer takes an optional second arg, so document this
* be a bit more explicit that the arguments to rate are in bytes
 1.65 30-Sep-1999  lukem * In the !NI_NUMERICHOST case (i.e, getaddrinfo() challenged systems), portnum
should be in host order. found/fixed by Matthias Pfaller <leo@dachau.marco.de>
* parse_url(): improve checking of portnum, and add an extra argument to pass
back the parsed portnum to the caller (reduces a bit of code duplication)
* Move the KAME/WIDE copyrights after the BSD/TNFi ones. Since there was
significantly less code added under the former, it's only fair on the latter.
 1.64 30-Sep-1999  lukem * fix initialisation of home[]
* fetch_url(): if path would be NULL, return strdup("")
 1.63 28-Sep-1999  lukem * add new commands:
lpage page local files
pdir as dir, but through your $PAGER
pls as ls, but through your $PAGER
* implement docase() (a la dotrans() et al) and use appropriately, rototilling
some duplicated code
* globulize(): modify to return a pointer to the strdup()ed result in all cases,
and hack the code that calls it to take this into account
* replace strcpy() and strncpy() with strlcpy()
* put(), getit(): use some aptly named local vars instead of argv[...]
* delint
 1.62 27-Sep-1999  lukem * idle(): rename to idlecmd(). certain linux distributions have an incompatible
prototype for idle() in <unistd.h> (which i thought was against namespace
and sensibility guidelines, but...)
* consistently use xsignal() instead of signal(). we get known behaviour
in all cases (SA_RESTART), which is good for some borken foreign systems.
* remove signal.h from most files; it's unnecessary now
* fetch_url(): use `long chunksize' instead of ssize_t; it's more portable, and
we're setting chunksize with strtol() anyway
* xsignal(): only use SA_RESTART if it exists. SunOS 4.x doesn't have it
but has the inverse (SA_INTERRUPT). the original function i was inspired
from had this support (lib/signal.c, W. Richard Stevens' `UNP 2nd ed Vol 1').
* remove <termios.h> from util.c; it should be unnecessary now
 1.61 24-Sep-1999  lukem * use %lld instead of %qd to print out (long long) vars.
(slightly more portable; e.g, solaris supports this)
* remove some fluff (lint)
 1.60 24-Sep-1999  lukem use an off_t instead of quad_t (or long #ifndef NO_QUAD) to store the
temporary restart point. (yet another portability fix...)
 1.59 24-Sep-1999  lukem * finish replacing snprintf() with sprintf(), for portability reasons.
add lots of comments about how to size up the buffers, and add extra
checks to hopefully ensure that there won't be an overflow (unless
someone modifies the length of the sprintf()s).
* as part of the above, slightly rework the way the `*' bar is calculated.
also fixes a display bug when > 160 stars were needed to be printed.
the maximum progress bar width at this time is 256.
* remove some code that checks the port that was #if 0-ed out as part of the
ipv6 migration; it's not going to be used again.
 1.58 22-Sep-1999  lukem branches: 1.58.2;
* add support for `xferbuf', which sets both `sndbuf' and `rcvbuf'
* document the above three commands
* rototill the way the sndbuf and rcvbuf work. remove resetsockbufsize()
* use the appropriate socket buffer size as the size of the buffer that
the read()/write() loops use. speeds up things in some cases.
 1.57 22-Sep-1999  lukem replace snprintf() with strlcpy(), strlcat(), or direct assignment
where appropriate. (strlc*() are easier to port to foriegn systems).

XXX; there's still a few snprintf's in the progress meter stuff to convert
 1.56 21-Sep-1999  lukem reword EPSV status line
 1.55 11-Jul-1999  itojun add epsv4 command, which enable/disable the use of EPSV/EPRT.

this is mainly for (hypothetical) ftp server which disconnect clients
that use EPSV/EPRT. I've never seen any ftp server like this, but
epsv4 command may be of use when such an ftp server is found.
 1.54 02-Jul-1999  itojun add dual-stack (IPv4/v6) support. hope I broke no other part...
 1.53 29-Jun-1999  lukem s/xfer/transfer/
 1.52 29-Jun-1999  lukem [fear this; more ftp hacking from lukem :-]

features:
 1.51 20-Jun-1999  cgd split the "SMALL" #ifdefs into ones for NO_ABOUT, NO_EDITCOMPLETE, and
NO_PROGRESS. -DSMALL still implies all of those. progress meter support
isn't necessary for the smallest possible ftp client, but it adds very
little space and makes users' lives much better. Therefore, it should
be enabled for installation media if at all possible.
 1.50 11-Jun-1999  lukem use default pager if $PAGER=""
 1.49 02-Jun-1999  lukem * fix gate mode to login as `user@realhost' rather than using PASSERVE;
the latter only seemed to work for TIS Gauntlet and not TIS fwtk.
thanks to simonb@netbsd.org for testing this. fixes [bin/5556].
* if EOF (e.g, ^D) is entered at a username/password/account prompt which
happens to use fgets(), exit the login rather than treating EOF as CR.
* don't use the comma operator where separate statements are valid
* always use snprintf to copy stuff into malloced buffers, just in case
typos creep in and mean that the buffer ends up being overflowed
 1.48 12-May-1999  lukem update copyright
 1.47 08-Mar-1999  lukem branches: 1.47.2;
whitespace & copyright fixes
 1.46 24-Jan-1999  lukem * -v enables verbose & progress, -V disables both
* set setvbuf(ttyout, NULL, _IOLBF, 0) and remove a bunch of fflush(ttyout).
* use fwrite() instead of write() for progressmeter (don't intermix stdio
with non stdio ops)
 1.45 24-Jan-1999  lukem flush ttyout in togglevar()
 1.44 08-Oct-1998  lukem * hash & progress are mutually exclusive. [noted by mrg@netbsd.org]
* in autofetch mode, don't attempt to 'cd /' on first xfer (only on
subsequent xfers), as some ftp daemons don't permit that.
[noted by dbj@netbsd.org].
XXX: code could be smarter (but a lot more complicated to boot); this
works for now.
 1.43 01-Sep-1998  lukem replace command(buf) with command("%s", buf) in quote1(), thus preventing
'quote' commands that contain % formats from being mis-intepreted.
noted by Lundman <lundman@argonaut.com> on current-users@netbsd.org
 1.42 08-Aug-1998  lukem use <limits.h> for INT_MAX (more portable)
 1.41 03-Aug-1998  lukem features:
* support $no_proxy, which is a comma or space separated list of
host[:port] elements for which proxying is to be disabled.
(asked for by cgd in [bin/5027])
* if $FTPANONPASS is defined, use that as the anon ftp password
(instead of "`whoami`@")
* allow http URL's without a filename as long as an output file
is specified.

other stuff:
* implement parse_url(), which breaks up a URL into its bits, and use.
* simplify url_get() and auto_fetch() to use parse_url() and to not
modify the supplied URL or a copy of it.
* implement xmalloc() and xstrdup(); error-checked malloc()/strdup()
* add more consistency to messages, quoting strings in output as `%s'
 1.40 26-Jul-1998  lukem * spell `retrieved' correctly!
* parse Last-Modified: HTTP headers for date, and change time of
retrieved file if parsing succeeded
* support file:/// and file://localhost/ URLs
* in url_get(), re-write byte moving code to consistently use fread()
and fwrite(), and check errors at end with ferror()
* add about:* easter-egg (#ifndef SMALL :-)
 1.39 10-Jul-1998  thorpej Add "sndbuf" and "rcvbuf" commands for setting the socket buffer sizes,
which in turn can allow the use of larger TCP windows. This is a work in
progress; there is not yet support for specifying global defaults or
user prefrences on a host/network basis.
 1.38 19-Jun-1998  kleink GLOB_QUOTE is dead; per POSIX, backslash quoting of special characters being
enabled is the default behaviour.
 1.37 04-Jun-1998  lukem some fixes & enhancements from openbsd's ftp, with extra fixes by me:
* default to passive with active fallback. $FTPMODE modifies this behaviour.
-A forces active connection.
* support '-o outfile' for auto-fetched files. outfile can be a file,
`-' (for stdout), or '|command' (to output each file through command).
* support '-r waittime', which retries the connection after waittime seconds
if it fails.
* fix 'page file' when restart is non-zero.
* try all ip-addresses of a host in a http fetch (as the normal ftp fetch
does).

XXX: a ``broken pipe'' error sometimes occurs with -o '|command';
i haven't tracked this down yet.
 1.36 20-May-1998  christos - add <signal.h> since we are using signals.
- cast arg to is*() to unsigned char.
- don't require quad_t to exist to compile.
 1.35 01-Apr-1998  kleink Need <time.h> for asctime() and localtime() prototypes.
 1.34 18-Jan-1998  lukem define MAX_IN_PORT_T and use instead of USHRT_MAX. suggested by cgd@netbsd.org
 1.33 18-Jan-1998  lukem * ensure buffer for username is initialised, so ^D on username prompt
doesn't use garbage for the username. from "Soren S. Jorvang" <soren@t.dk>
in [bin/4559]
* use in_port_t for ports, and USHRT_MAX instead of 0xffff
(from millert@openbsd.org)
* use `NULL' instead of `(.... *)0' where appropriate.
 1.32 02-Nov-1997  lukem correctly detect good vs bad hash mark values. (found by <enami@netbsd.org>)
 1.31 01-Nov-1997  lukem * in recvrequest(), ignore restart_point unless "RETR"ieving. fixes problems
where a remote completion or `mget' would confuse the client a `restart'
had been issued beforehand. now, `restart' is remembered until an operation
that can actually use it is invoked.
* in sendrequest(), don't reset restart_point upon entry. fixes `restart'
for `put' operations.
* if `restart' is invoked with no arguments, print current setting instead
of displaying a usage
* consistently use printf("%qd", (long long)restart_point) when displaying
restart_point
* use strto[lq]() instead of atol() when parsing `mark' and `restart' values
* remove unnecessary strlen()s when result of previous snprintf() will do
* replace a few malloc()/strcpy()s with strdup()s
* use SECSPERHOUR instead of '3600'
 1.30 26-Sep-1997  lukem branches: 1.30.2;
add a space to restart message. from geoff wing in [bin/4161]
 1.29 21-Sep-1997  lukem More cleanups from Todd Miller <Todd.Miller@courtesan.com>:
* Use an int, not "union wait".
* Move the "parsed_url" label so that the next statement is not an "else" --
some ansi C compilers don't like it the old way (SGI's for example).
* Deal with the possibility of getlogin(2) failing.
* Don't error out if the remote server doesn't support the "MDTM" command.
 1.28 13-Sep-1997  lukem Fixes from Todd Miller <Todd.Miller@courtesan.com>:
* use size_t instead of int in places
* use symbolic constants when using access()
 1.27 18-Aug-1997  lukem bugs fixed:
* don't interpret '-' or '|' when a local filename is determined from
the remote name (i.e, in mget, and in get with only one argument).
This is implemented using an extra argument to recvrequest().
Fixes a major security hole.
* clean up memory leak when using globulize()
* clean up a couple of comments
* fix wording in TNF copyright

features added:
* support for TIS fwtk gate-ftp servers:
* read defaults from $FTPSERVER && $FTPSERVERPORT
* start in gate-ftp mode if invoked as 'gate-ftp'
* toggle or set with 'gate [host [port]]'
 1.26 21-Jul-1997  lukem fix compile on alpha:
* cast all %qd printfs to (long long) [inspired by billc@warped.net]
* replace sscanf %qd with strtoq() et al
 1.25 20-Jul-1997  lukem * use RCSID() && COPYRIGHT() macros
* cleanup for WARNS=1 (including some ugly '(void)&var' bits wrapped in
#ifdef __GNUC__ to shut up gcc warnings WRT setjmp/longjmp)
* use strtol() instead of atoi(), and more extensively check result of
conversion
* use u_int16_t instead of short or int for TCP port addresses
 1.24 17-May-1997  pk NULL => 0 (Arne Juul; PR#3629)
 1.23 14-Apr-1997  lukem More enhancements/bugfixes (when will it end?)
* differentiate between being connected, and being logged in
* cleanup some text messages
* support username & password ftp URLs (ftp://user:pass@host/) in non-proxy
situations; assume proxy supports it for proxy situations.
* cd to / before performing any autofetch transfers
* use strncasecmp in URL parsing. fix from <Todd.Miller@courtesan.com>
 1.22 05-Apr-1997  lukem * support $ftp_proxy for ftp:// transfers [bin/3245]
* add "more" & "less" as synonyms for "page"
* move editline setup code into controlediting(), and call appropriately.
only setup setup terminal if going into interactive mode. inspired
by Todd Miller <todd.miller@courtesan.com>
 1.21 16-Mar-1997  lukem Fixes from <Todd.Miller@cs.colorado.edu>, with some cleanup and reworks by me:
* only echo "Passive mode" in verbose mode; scripts that use ftp
may get unwanted output otherwise
* disable progress bar and modification time preservation when
retreiving to a non-regular files. fixes progress bar getting in
way of "get file /dev/tty"
* setup el_init() et al if editing is set, not if fromatty.
TODO: migrate this to a function, and call if editing is turned on later
in the session. also implement edit_cleanup if editing is turned off
* call el_set() after setting SIGWINCH handler. This fixes the problem
when suspending in a non-cbreak shell (e.g, csh) would trash your tty mode.
* reset interactive mode correctly in auto_fetch() mget mode
 1.20 14-Mar-1997  christos SMALLFTP->SMALL; remove SMALL from Makefile
 1.19 13-Mar-1997  lukem Features:
* support remglobbing of auto_fetch arguments
* new flag - '-e'; disable editing
* "page file" == "get file |${PAGER-less}"

Bugfixes/cleanup:
* consistently use a trailing '.' on messages
* code cleanup, including buffer overrun fixes, use puts
and putchar in places, etc (inspired by OpenBSD mods)
* disable progress bar when local-file is a pipe or '-'
* skip \r in http headers
* fix remote ftpd slash bug more elegantly (so it works with ////)
* abort_remote(): check if cout==NULL before using it. should fix [bin/3273]
* fixed up cosmetic problems when complete_remote() generated errors from the
remote server (such as "no files found", "login with user and pass", ...)
done by adding extra argument to remglob(), which is a pointer to an error
buffer to put messages in rather than printing to stdout.
 1.18 01-Feb-1997  lukem [Yet Another Huge Ftp Commit - hopefully the last for a while,
barring any more little things people want added ...]

New features:
* progressmeter is now asynchronous, so "stalled" transfers can be
detected. "- stalled -" is displayed instead of the ETA in this case.
When the xfer resumes, the time that the xfer was stalled for is
factored out of the ETA. It is debatable whether this is better than
not factoring it out, but I like it this way (I.e, if it stalls for 8
seconds and the ETA was 30 seconds, when it resumes the ETA will still
be 30 seconds).
* verbosity can be disabled on the command line (-V), so that in auto-fetch
mode the only lines displayed will be a description of the file, and
the progress bar (if possible)
* if the screen is resized (and detected via the SIGWINCH signal), the
progress bar will rescale automatically.

Bugs fixed:
* progress bar will not use the last character on the line, as this can
cause problems on some terminals
* screen dimensions (via ioctl(TIOCWINSZ)) should use stdout not stdin
* progressmeter() used some vars before initialising them
* ^D will quit now. [fixes bin/3162]
* use hstrerror() to generate error message for host name lookup failure.
* use getcwd instead of getwd (it should have been OK, but why tempt fate?)
* auto-fetch transfers will always return a positive exit value upon failure
or interruption, relative to the file's position in argv[].
* remote completion of / will work, without putting a leading "///".
This is actually a bug in ftpd(1), where "NLST /" prefixes all names
with "//", but fixing every ftpd(1) is not an option...
 1.17 19-Jan-1997  veego Add missing ;
 1.16 19-Jan-1997  lukem New features:
* Command line editing via editline(3) library.
* Context sensitive command and file completion, including remote files.

Enhancements to auto-fetch feature:
* Support for http:// URLs using the http protocol, including proxy HTTP
support via $htty_proxy if it's defined.
* The connection is kept open between successive files on the same host.
(obviously, this does not count for http requests.)
* Return value of ftp is 0 on no error, or the offset in argv[] of the file
which failed (i.e., argv[x] failed, ftp returns x).
* If the path in an ftp URL or classic format line has a trailing '/',
cd to the path and enter interactive mode. Fixes [bin/3011], albiet
requiring the user to help ftp in determining the operation.

Other changes:
* '-P port' works for normal ftp, and is the default for all classic style
auto-fetch transfers and for ftp URLs that don't specify the port.
(previously it would just work for the first xfer.)
* Some code moved into separated files along logical divisions.
* Editing and completion can be compiled out with -DSMALLFTP.
 1.15 09-Jan-1997  tls RCS ID police
 1.14 03-Jan-1997  lukem doproxy(): remove leading "proxy " from input buffer so if another() is called
and line[] is reparsed to build margc/margv, commands don't get confused.
 1.13 29-Dec-1996  lukem * preserve modtime if size is 0 [bin/3040, Enami Tsugutomo]
* in autofetch mode, don't retry a file if the connection failed - skip it
and move to the next file. [bin/3051, Matt Green]
* only print error messages from SIZE or MDTM if the user used 'size' or
'modt' (respectively). prevents extraneous warnings during normal transfers
when connected to a site which doesn't support these (behaviour prior to
last commit), but still allows error feedback to specific user requests
for this info (which the last commit broke).
* 'account': only accept one optional argument
* if invoked as pftp, default to passive mode on (from FreeBSD)
* remove leading '0 ' in progress bar - looked ugly
* use warn instead of perror
* use strncpy when src isn't known to have safe length
* remglob(): use mkstemp() to prevent symlink games, and don't override
_PATH_TMP, use it as the prefix to the temp file
 1.12 25-Dec-1996  christos - Don't print the '500 command not understood' reply for the SIZE and
MDTM commands which are not supported by all ftp servers, unless we
are in debugging mode.
 1.11 06-Dec-1996  lukem functionality mods:
* implement 'progress bar/meter' (inspired by ncftp). use 'progress' to
toggle on. it will display current file size to 5 digits, automatically
determining suffix (up to 16384 P (petabytes) == 2^64).
* 'ls' now uses NLST (unadorned listing), a la older ftp clients. 'dir'
still does LIST (long listing). idea from John Nemeth <jnemeth@cue.bc.ca>

bug fixes:
* return first line of reply in reply_string[] from getreply(), instead
of last line. This fixes [bin/741] (parsing of SYST), and also means
that SIZE and MDTM messages will be parsed correctly if they're longer
than 1 line.
* parse URL-style auto-ftps that have no filename correctly
(e.g, ftp://host, ftp://host/, ftp://host/dir/). pointed out by
Jaromir Dolecek <dolecek@saruman.ics.muni.cz>
* pass the correct size array in 2nd arg of utimes() when setting the
modification time
 1.10 28-Nov-1996  lukem More features, some of which were inspired by changes that
friedman@gnu.ai.mit.edu (Noah Friedman) made to his modified ftp:
- implement "lpwd" - local pwd
- implement "preserve" - toggle preserving of file modification
times on retrieved files
- allow for explicit "on" or "off" arg to toggle commands
- "exit" synonym for "quit", "msend" synonym for "mput"
- in confirmation mode, allow 'a' (yes to rest of current command),
and 'p' (turn off prompt mode, as if 'prompt off' was done,
effective immediately)
- "modtime" returns time formatted as localtime, not GMT

Bug fixes:
- check for extraneous args on commands
- cleanup const usage, line formatting
- create 0 length temporary file in remglob() to prevent symlink games
(from OpenBSD)
- check length of filename of ~/.netrc (from OpenBSD)
 1.9 25-Nov-1996  lukem cleanups/bugfixes:
- don't echo 'ACCT' parameter when debugging (a la 'PASS')
- Fix checking of directory access for "/foo", the parent
directory is "/", not "" (from FreeBSD)
- remove trailing whitespace on lines
- add any missing NetBSD tags
- cleanups to man page, includinging sorting options description

feature additions:
- variable sized hash marks (from [bin/683], but done in the hash command
as an optional arg)
- more user-friendly transfer time printing (from FreeBSD, with mods)
- '-p' command line option to jump into PASV mode (closes [bin/2857],
but with an option rather than checking argv[0])
- SIGINFO support for printing xfer stats when sending/receiving requests
- '-P port' for changing the port to connect to (from thorpej@netbsd.org)
- '-a': bypass normal login, and try anonymous login (from OpenBSD
via thorpej)
- autofetch files via url (ftp://...) or "classic" (host:/file)
(from OpenBSD via thorpej)
- 'ftp' synonymous with 'open' (from FreeBSD)
 1.8 08-Sep-1995  tls Sync with 4.4lite2
 1.7 15-Dec-1994  jtc Null terminate case-mapped filenames (PR #640).
 1.6 29-Aug-1994  mycroft branches: 1.6.2;
Add RCS ids.
 1.5 25-Aug-1994  cgd passive mode support, from David Carrel <carrel@cisco.com>, and cleaned
up for the new ftp sources by me.
 1.4 25-Aug-1994  cgd clean up import, no local changes.
 1.3 27-Mar-1994  cgd off_t foo
 1.2 01-Aug-1993  mycroft Add RCS identifiers.
 1.1 21-Mar-1993  cgd branches: 1.1.1; 1.1.2;
Initial revision
 1.1.2.1 25-Aug-1994  cgd import from 4.4-Lite
 1.1.1.2 05-Sep-1995  tls imported from 44lite2
 1.1.1.1 21-Mar-1993  cgd initial import of 386bsd-0.1 sources
 1.6.2.2 29-Aug-1994  mycroft Add RCS ids.
 1.6.2.1 29-Aug-1994  mycroft file cmds.c was added on branch netbsd-1-0 on 1994-08-29 03:09:07 +0000
 1.30.2.2 10-Nov-1998  cgd pull up rev(s) 1.33-1.44 from trunk. (feyrer)
 1.30.2.1 18-Nov-1997  mellon Pull rev 1.31 and 1.32 up from trunk (lukem)
 1.47.2.3 25-Jun-1999  cgd pull up rev 1.51 from trunk. (cgd)
 1.47.2.2 23-Jun-1999  perry pullup 1.49->1.50 (kleink)
 1.47.2.1 22-Jun-1999  perry pullup 1.47->1.49 (lukem)
 1.58.2.1 27-Dec-1999  wrstuden Pull up to last week's -current.
 1.85.2.1 23-Jun-2000  minoura Sync w/ netbsd-1-5-base.
 1.87.2.1 18-Oct-2000  tv Pullup usr.bin string format fixes [is].
See "cvs log" for explicit revision numbers per file, from sommerfeld.
 1.111.2.5 24-Jul-2005  tron Pull up revision 1.116 (requested by lukem in ticket #606):
Add NO_USAGE and NO_DEBUG so that we can fit in the floppies again.
 1.111.2.4 24-Jul-2005  tron Pull up revision 1.115 (requested by lukem in ticket #606):
Implement:
int getline(FILE *stream, char *buf, size_t buflen, const char **errormsg)
Read a line from the FILE stream into buf/buflen using fgets(), so up
to buflen-1 chars will be read and the result will be NUL terminated.
If the line has a trailing newline it will be removed.
If the line is too long, excess characters will be read until
newline/EOF/error.
Various -ve return values indicate different errors, and errormsg
will be changed to an error description if it's not NULL.
Convert to use getline() instead of fgets() whenever reading user input
to ensure that an overly long input line doesn't leave excess characters
for the next input operation to accidentally use as input.
Zero out the password & account after we've finished with it.
Consistently use getpass(3) (i.e, character echo suppressed) when
reading the account data. For some reason, historically the "login"
code suppressed echo for Account: yet the "user" command did not!
Display the hostname in the "getaddrinfo failed" warning.
Appease some -Wcast-qual warnings. Fixing all of these requires
significant code refactoring. (mmm, legacy code).
 1.111.2.3 24-Jul-2005  tron Pull up revision 1.114 (requested by lukem in ticket #606):
Use size_t instead of int where appropriate.
 1.111.2.2 24-Jul-2005  tron Pull up revision 1.113 (requested by lukem in ticket #606):
Some const cleanups.
 1.111.2.1 09-May-2005  tron Pull up revision 1.112 (requested by lukem in ticket #266):
gratuitous whitespace cleanup (before someone else jumps the gun...)
 1.123.10.1 18-May-2008  yamt sync with head.
 1.124.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.127.2.1 13-May-2009  jym Sync with HEAD.

Third (and last) commit. See http://mail-index.netbsd.org/source-changes/2009/05/13/msg221222.html
 1.132.2.2 23-Jan-2013  yamt sync with head
 1.132.2.1 17-Apr-2012  yamt sync with head
 1.134.6.1 25-Feb-2013  tls resync with head
 1.134.2.2 27-Aug-2016  bouyer Apply patch, requested by nonaka in ticket #1375:
src/usr.bin/ftp/cmds.c: patch
src/usr.bin/ftp/fetch.c: patch
src/usr.bin/ftp/ftp.1: patch
src/usr.bin/ftp/ftp.c: patch
src/usr.bin/ftp/ftp_var.h: patch
src/usr.bin/ftp/main.c: patch
src/usr.bin/ftp/ssl.c: patch
src/usr.bin/ftp/ssl.h: patch
src/usr.bin/ftp/version.h: patch
Update ftp(1) to version 20150912, adding https via proxy support.
 1.134.2.1 17-Dec-2013  bouyer Apply patch, requested by tron in ticket #997:
usr.bin/ftp/Makefile patch
usr.bin/ftp/cmds.c patch
usr.bin/ftp/cmdtab.c patch
usr.bin/ftp/extern.h patch
usr.bin/ftp/fetch.c patch
usr.bin/ftp/ftp.1 patch
usr.bin/ftp/ftp.c patch
usr.bin/ftp/ftp_var.h patch
usr.bin/ftp/main.c patch
usr.bin/ftp/progressbar.c patch
usr.bin/ftp/ssl.c patch
usr.bin/ftp/ssl.h patch
usr.bin/ftp/util.c patch
usr.bin/ftp/version.h patch

Add HTTPS support to ftp(1).
 1.135.8.1 13-Mar-2016  martin Pull up following revision(s) (requested by nonakap in ticket #1133):
usr.bin/ftp/fetch.c: revision 1.208-1.221
usr.bin/ftp/cmds.c: revision 1.136-1.137
usr.bin/ftp/ssl.c: revision 1.5
usr.bin/ftp/ftp.c: revision 1.165-1.166
usr.bin/ftp/ftp_var.h: revision 1.84

Workaround const issues of SSL_set_tlsext_host_name.

Use the proper format "[IPv6 address]:port" when reporting connection
attempts to IPv6 endpoints.

(Hopefully) fix build without IPv6 support

Try to factor out some code, this is completely out of control.

Separate no_proxy handling.

Factor the proxy handling code out.

Fix compile failure without WITH_SSL.

PR/50438: NONAKA Kimihiro: ftp(1): CONNECT method support

make DPRINTF/DWARN always statements.

Fix to connect https via proxy.

Fix ttyout message.

Simplify and factor out connect message

Split the position/size parsing into a separate function.

Mark function as only needed with ssl.

Fix downloads of local files using file:// URLs

Initialize the token match pointer.

use sizeof() and array notation.
CID 1354295: Array overrun.
 1.137.8.3 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1763:

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/cmds.c up to 1.141
usr.bin/ftp/complete.c up to 1.47
usr.bin/ftp/domacro.c up to 1.23
usr.bin/ftp/extern.h up to 1.82
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp.c up to 1.174
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/progressbar.c up to 1.24
usr.bin/ftp/progressbar.h up to 1.9
usr.bin/ftp/ssl.c up to 1.12
usr.bin/ftp/ssl.h up to 1.5
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

ftp(1): validate address from PASV and LPSV response.
ftp(1): use raw write(2) instead of fwrite(3) to avoid stream
corruption because of the progress bar interrupts.
Fixes for PR 56219 and PR 55857.
PR 57003: Support relative redirects.
 1.137.8.2 12-Sep-2022  martin Backout ticket #1763 for now - trust anchors are not solved.
 1.137.8.1 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1763:

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/cmds.c up to 1.141
usr.bin/ftp/complete.c up to 1.47
usr.bin/ftp/domacro.c up to 1.23
usr.bin/ftp/extern.h up to 1.82
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp.c up to 1.174
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/progressbar.c up to 1.24
usr.bin/ftp/progressbar.h up to 1.9
usr.bin/ftp/ssl.c up to 1.11
usr.bin/ftp/ssl.h up to 1.5
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

ftp(1): validate address from PASV and LPSV response.
ftp(1): use raw write(2) instead of fwrite(3) to avoid stream
corruption because of the progress bar interrupts.
Fixes for PR 56219 and PR 55857.
PR 57003: Support relative redirects.
 1.138.4.1 10-Jun-2019  christos Sync with HEAD
 1.140.2.2 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1523

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/ssl.c up to 1.12
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

PR 57003: Support relative redirects.
 1.140.2.1 14-Jun-2021  martin Pull up following revision(s) (requested by lukem in ticket #1294):

usr.bin/ftp/cmds.c: revision 1.141
usr.bin/ftp/ftp.1: revision 1.143

ftp(1): fix description of "debug"

"debug" command and documentation got accidentally renamed
to "ftp_debug" 13 years ago, and was only partially fixed.
 1.141.8.1 02-Aug-2025  perseant Sync with HEAD
 1.141.6.2 02-Dec-2024  martin Pull up following revision(s) (requested by lukem in ticket #1021):

usr.bin/ftp/ftp.c: revision 1.178
usr.bin/ftp/version.h: revision 1.99
usr.bin/ftp/ruserpass.c: revision 1.35
usr.bin/ftp/main.c: revision 1.134
usr.bin/ftp/main.c: revision 1.135
usr.bin/ftp/progressbar.c: revision 1.27
usr.bin/ftp/util.c: revision 1.165
usr.bin/ftp/cmds.c: revision 1.144
usr.bin/ftp/extern.h: revision 1.84
usr.bin/ftp/fetch.c: revision 1.242
usr.bin/ftp/ftp.1: revision 1.160

s/bninary/binary/ in comment.

extract duplicate code into a function.

Check bounds when copying to destination.

Remove const where the const string ended up being overwritten.

use unsigned when doing shifts.

remove const when string gets overwritten.

ftp: exit non-zero if short http transfer when filesize is known
If a http file size is known and the fetch finishes with less bytes
transferred, exit non-zero.
Bump version to 20241129.
PR bin/54713
PR bin/58281

ftp: help improvements
Document -? as a separate mode.
Document -H HEADER in the usage.
Clarify units for -b and -x.
Consistent argument names between ftp -? and ftp(1).

ftp: order getopt Upper before lower
Consistently order options in getopt and the switch
with the upper case option before the lower case option.
This makes it easier to cross-reference with -? and ftp(1).
No functional change.
 1.141.6.1 13-Oct-2024  martin Pull up following revision(s) (requested by riastradh in ticket #970):

tests/usr.bin/Makefile: revision 1.42
usr.bin/ftp/ruserpass.c: revision 1.34
usr.bin/ftp/main.c: revision 1.130
usr.bin/ftp/ssl.c: revision 1.17
usr.bin/ftp/main.c: revision 1.131
usr.bin/ftp/ssl.c: revision 1.18
usr.bin/ftp/main.c: revision 1.132
usr.bin/ftp/ssl.c: revision 1.19
usr.bin/ftp/main.c: revision 1.133
distrib/sets/lists/tests/mi: revision 1.1342
usr.bin/ftp/ftp.1: revision 1.151
usr.bin/ftp/ftp.1: revision 1.152
usr.bin/ftp/progressbar.c: revision 1.25
usr.bin/ftp/ftp.1: revision 1.153
usr.bin/ftp/progressbar.c: revision 1.26
usr.bin/ftp/ftp.1: revision 1.155
usr.bin/ftp/ftp.1: revision 1.156
usr.bin/ftp/fetch.c: revision 1.239
usr.bin/ftp/ftp.1: revision 1.157
usr.bin/ftp/ftp.1: revision 1.158
usr.bin/ftp/ftp.1: revision 1.159
usr.bin/ftp/ftp_var.h: revision 1.87
etc/mtree/NetBSD.dist.tests: revision 1.208
usr.bin/ftp/ftp_var.h: revision 1.88
usr.bin/ftp/ftp_var.h: revision 1.89
usr.bin/ftp/cmds.c: revision 1.142
usr.bin/ftp/util.c: revision 1.168
usr.bin/ftp/cmds.c: revision 1.143
tests/usr.bin/ftp/custom_headers.sh: revision 1.1
usr.bin/ftp/ssl.c: revision 1.20
usr.bin/ftp/complete.c: revision 1.48
tests/usr.bin/ftp/Makefile: revision 1.1
tests/usr.bin/ftp/t_custom_headers.sh: revision 1.1
usr.bin/ftp/fetch.c: revision 1.240
usr.bin/ftp/fetch.c: revision 1.241
usr.bin/ftp/ftp.c: revision 1.176
usr.bin/ftp/ftp.c: revision 1.177
(all via patch)

ftp(1): wording and formatting improvements

Fix grammar issue with "Support values" reported in private mail.
Document all file transfer types in "type" and cross-reference that.
Consistency fixes in describing file transfer parameters and types.

Fix some mandoc -Tlint issues (except "useless macro: Tn").

Add -b <buflen> to specify the buffer size.

ftp: bump FTPBUFLEN from 4kB to 16kB
sourceforge.net returns a 5kB content-security-policy.
Analyzed by mlelstv@ who reports usual limits are between 4kB and 48kB.
default is now 16K

ftp: improve -b documentation

Order -b bufsize in the synopsis.

Document the actual default value.

ftp: improve units used in comments and errors
Use "KiB" instead of "K" in errors.
Clarify related comments.

pass some lint.

PR/58581: Sunil Nimmagadda: Add flag to allow specifying extra http header
fields.

ftp(1): Nix trailing whitespace in man page.
No functional change intended.

PR bin/58581: ftp(1) should allow specifying header fields in http requests
fix markup (h -> H), explain about multiple headers, fix usage (from RVP)

Don't forget the dot, use the intended macro name (I think),
and improve the wording a little. (All related to the -H option.)
ftp(1): Add test for custom HTTP header fields.

Based on a patch from Sunil Nimmagadda.

PR bin/58581: ftp(1) should allow specifying header fields in http
requests
 1.53 25-Feb-2023  mlelstv Add option sslnoverify to control validation of SSL certificates.
Add netrc processing to fetch-mode (URL on command line) to enable options and autologin
via netrc.
Fix SSL cleanup in some error paths.

Certificate validation is now enabled by default. Set FTPSSLNOVERIFY=1 in environment
or configure a corresponding init macro via netrc to not validate certs (required if
you haven't installed a required CA certificate for OpenSSL).

Discussed with lukem@ on icb.
 1.52 22-Dec-2012  christos branches: 1.52.40;
document https support, mentioning certificates are not validated, and obey
https_proxy.
 1.51 12-Apr-2009  lukem branches: 1.51.6; 1.51.8; 1.51.12;
Fix numerous WARNS=4 issues (-Wcast-qual -Wsign-compare).
 1.50 30-Sep-2008  lukem branches: 1.50.6;
update copyrights
 1.49 10-May-2008  skd Add epsv6 and epsv to disable extended passive mode for ipv6 or both ipv4 and ipv6 respectively. This hack is due to our friends a Juniper Networks who break
epsv in ipv6. Should be fixed in ScreenOS 6.2.X.
 1.48 28-Apr-2008  martin branches: 1.48.2;
Remove clause 3 and 4 from TNF licenses
 1.47 11-Apr-2007  lukem branches: 1.47.10;
Revert rename of "debug" command from "ftp_debug" back to "debug".
 1.46 31-Jan-2006  christos rename debug to ftp_debug. grr libssh.
 1.45 09-Jun-2005  lukem copyright maintenance
 1.44 11-Apr-2005  lukem gratuitous whitespace cleanup (before someone else jumps the gun...)
 1.43 15-Jul-2004  lukem branches: 1.43.2;
Fix bug in Christos' recent cleanup which broke the "help" and "rhelp" commands.
 1.42 06-Jun-2004  christos Save approximately 8K by not including http authentication, extended status
messages and help strings when the appropriate options are set.
 1.41 07-Aug-2003  agc Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22365, verified by myself.
 1.40 29-Jul-2002  lukem Improve description of help text for "passive".
Noted by Yar Tikhiy <yar at FreeBSD.org> in private email.
 1.39 15-Nov-2000  lukem - implement "mreget"; as per "mget" but uses "reget" instead of "get"
- add -N netrc and $NETRC, as methods to select an alternative .netrc file
- cache local user name and home directory for further use
- in mget(), use docase() instead of a local version to do the case
conversion.
 1.38 14-Sep-2000  lukem since everything else here uses ANSI C, we might as well replace __STRING()
with the ANSI C stringization stuff...
 1.37 28-Jul-2000  lukem rename "opts" to "remopts", so people used to "o host" don't get bitten...
 1.36 18-Jul-2000  lukem add support for FEAT and OPTS commands with `features' and `opts'.
(from RFC 2389).

add support for MLST & MLSD (machine parseble listings) with 'mlst', 'mlsd'
and 'pmlsd' (mlsd |$PAGER) commands. (from draft-ietf-ftpext-mlst-11)

rename remotesyst() to getremoteinfo(), and modify to parse the result from
FEAT (if supported), and take into account the support for the various
extensions such as MDTM, SIZE, REST (STREAM), MLSD, and FEAT/OPTS.
put each feature into one of the following categories:
- known to work (explicit FEAT)
- unknown but assume works until explicit failure, when it's
then tagged as `known not to work'.
- known not to work (FEAT succeeded but didn't return anything,
or was unknown and then explicit failure)
assign results into features[] matrix.

add support to getreply() so that an optional callback will be called
for each line received from the server except for the first and last.
this is used in FEAT (and MLST) parsing.

modify various commands to check if REST (STREAM), MDTM and SIZE are
explicitly or implicitly supported before using.

fix `syst' when verbose is off.

minor knf (indent goto labels by one space, etc).

simply various command usage handlers by assuming that argv != NULL except
for quit() and disconnect().
 1.35 01-May-2000  lukem * Add support for `fget localfile', which reads a list of filenames to
retrieve from localfile. Based on work by Darren Reed.
* Crank version.
* Update copyright dates.
 1.34 12-Nov-1999  lukem - implement `set rprompt'; right side version of `set prompt'. depends on
EL_RPROMPT support i added to editline(3).
- allow $FTPPROMPT and $FTPRPROMPT to override defaults for the relevant
prompts
- move `%' formatting code from prompt() to expandbuf().
- implement `%.' and `%c', similar to the same % codes in tcsh(1)
(functionality I added to tcsh nearly 6 years ago), except that `%.'
always does `...trailing' and `%c' always does `/<x>trailing'.
- unknown `%foo' codes get printed as `%foo'
 1.33 11-Nov-1999  lukem - implement updateremotepwd(); update the global variable `remotepwd'
to contain the remote working directory.
- add `set prompt', a user configurable prompt. (defaults to `ftp> ').
the following escape characters a la tcsh(1) are supported: %/, %m,
%M, and %n.
- add global var `username'; used by prompt code
- fix a couple of minor memory leaks
- bump version
 1.32 24-Oct-1999  lukem new features:
- add `usage'; displays the usage of a command.
implemented by calling the c_handler() with argc = 0, argv = "funcname".
- add `passive auto'; does the same as $FTPMODE=auto.
- add `set [option value]'; display all options, or set an option to a value.
- add `unset option'; unset an option.
- add getoptionvalue() to retrieve an option's value, and replace a few
global variables with calls to this.
- implement cleanuppeer(), which resets various bits of state back to
`disconnected'. call in disconnect() and lostpeer().
- support completing on `options'.
- improve recovery after a SIGINT may have closed the connection.
XXX: there's still a couple to fix

other stuff:
- various consistency fixes in the man page.
- ensure that the command usage strings in the code and man page match reality.
- mput/mget: check that the connection still exists before each xfer.
- minor cosmetic changes in confirm().
- set code correctly in sizecmd() and modtime()
- don't need \n in err() strings.
- change lostpeer to take an argument (rather than casting (sig_t)lostpeer
in signal handlers)
- knf and whitespace police.
 1.31 05-Oct-1999  lukem add TNFi copyright to all files i've done more than a minor amount of work to...
 1.30 05-Oct-1999  lukem enhancments from Marc Horowitz <marc@mit.edu> to improve connection timeouts:
* implement xsignal_restart(), which only sets the SA_RESTART flag if
specifically requested
* xsignal() is now a wrapper to xsignal_restart(). INFO, USR1, USR2 and WINCH
are restartable, ALRM, INT, PIPE and QUIT are not.
* improve getreply()'s timeout code to take advantage of the above.

other changes:
* improve wording of how globbing works for `classic' URLs (host:path).
suggested by John Refling <johnr@imageworks.com> in relation to PRs
[bin/8519] and [bin/8520]
* always compile in the `edit' command even if NO_EDITCOMPLETE defined.
it's just a no-op in the latter case, which is more consistent to
the users.
* always compile in about: support (i.e, remove NO_ABOUT).
i'm entitled to some vanity in this program...
* clean up some whitespace
 1.29 01-Oct-1999  lukem * newer takes an optional second arg, so document this
* be a bit more explicit that the arguments to rate are in bytes
 1.28 28-Sep-1999  lukem * add new commands:
lpage page local files
pdir as dir, but through your $PAGER
pls as ls, but through your $PAGER
* implement docase() (a la dotrans() et al) and use appropriately, rototilling
some duplicated code
* globulize(): modify to return a pointer to the strdup()ed result in all cases,
and hack the code that calls it to take this into account
* replace strcpy() and strncpy() with strlcpy()
* put(), getit(): use some aptly named local vars instead of argv[...]
* delint
 1.27 27-Sep-1999  lukem * idle(): rename to idlecmd(). certain linux distributions have an incompatible
prototype for idle() in <unistd.h> (which i thought was against namespace
and sensibility guidelines, but...)
* consistently use xsignal() instead of signal(). we get known behaviour
in all cases (SA_RESTART), which is good for some borken foreign systems.
* remove signal.h from most files; it's unnecessary now
* fetch_url(): use `long chunksize' instead of ssize_t; it's more portable, and
we're setting chunksize with strtol() anyway
* xsignal(): only use SA_RESTART if it exists. SunOS 4.x doesn't have it
but has the inverse (SA_INTERRUPT). the original function i was inspired
from had this support (lib/signal.c, W. Richard Stevens' `UNP 2nd ed Vol 1').
* remove <termios.h> from util.c; it should be unnecessary now
 1.26 22-Sep-1999  lukem branches: 1.26.2;
* add support for `xferbuf', which sets both `sndbuf' and `rcvbuf'
* document the above three commands
* rototill the way the sndbuf and rcvbuf work. remove resetsockbufsize()
* use the appropriate socket buffer size as the size of the buffer that
the read()/write() loops use. speeds up things in some cases.
 1.25 11-Jul-1999  itojun add epsv4 command, which enable/disable the use of EPSV/EPRT.

this is mainly for (hypothetical) ftp server which disconnect clients
that use EPSV/EPRT. I've never seen any ftp server like this, but
epsv4 command may be of use when such an ftp server is found.
 1.24 02-Jul-1999  itojun add dual-stack (IPv4/v6) support. hope I broke no other part...
 1.23 29-Jun-1999  lukem [fear this; more ftp hacking from lukem :-]

features:
 1.22 24-Jun-1999  christos include <signal.h> explicitly! we use sig_t extern.h, and we cannot
depend on other headers fetching it on other OS's.
 1.21 20-Jun-1999  cgd split the "SMALL" #ifdefs into ones for NO_ABOUT, NO_EDITCOMPLETE, and
NO_PROGRESS. -DSMALL still implies all of those. progress meter support
isn't necessary for the smallest possible ftp client, but it adds very
little space and makes users' lives much better. Therefore, it should
be enabled for installation media if at all possible.
 1.20 08-Mar-1999  lukem branches: 1.20.2;
whitespace & copyright fixes
 1.19 26-Jul-1998  lukem * spell `retrieved' correctly!
* parse Last-Modified: HTTP headers for date, and change time of
retrieved file if parsing succeeded
* support file:/// and file://localhost/ URLs
* in url_get(), re-write byte moving code to consistently use fread()
and fwrite(), and check errors at end with ferror()
* add about:* easter-egg (#ifndef SMALL :-)
 1.18 10-Jul-1998  thorpej Add "sndbuf" and "rcvbuf" commands for setting the socket buffer sizes,
which in turn can allow the use of larger TCP windows. This is a work in
progress; there is not yet support for specifying global defaults or
user prefrences on a host/network basis.
 1.17 18-Aug-1997  lukem branches: 1.17.2;
bugs fixed:
* don't interpret '-' or '|' when a local filename is determined from
the remote name (i.e, in mget, and in get with only one argument).
This is implemented using an extra argument to recvrequest().
Fixes a major security hole.
* clean up memory leak when using globulize()
* clean up a couple of comments
* fix wording in TNF copyright

features added:
* support for TIS fwtk gate-ftp servers:
* read defaults from $FTPSERVER && $FTPSERVERPORT
* start in gate-ftp mode if invoked as 'gate-ftp'
* toggle or set with 'gate [host [port]]'
 1.16 20-Jul-1997  lukem * use RCSID() && COPYRIGHT() macros
* cleanup for WARNS=1 (including some ugly '(void)&var' bits wrapped in
#ifdef __GNUC__ to shut up gcc warnings WRT setjmp/longjmp)
* use strtol() instead of atoi(), and more extensively check result of
conversion
* use u_int16_t instead of short or int for TCP port addresses
 1.15 05-Apr-1997  lukem * support $ftp_proxy for ftp:// transfers [bin/3245]
* add "more" & "less" as synonyms for "page"
* move editline setup code into controlediting(), and call appropriately.
only setup setup terminal if going into interactive mode. inspired
by Todd Miller <todd.miller@courtesan.com>
 1.14 14-Mar-1997  christos SMALLFTP->SMALL; remove SMALL from Makefile
 1.13 13-Mar-1997  lukem Features:
* support remglobbing of auto_fetch arguments
* new flag - '-e'; disable editing
* "page file" == "get file |${PAGER-less}"

Bugfixes/cleanup:
* consistently use a trailing '.' on messages
* code cleanup, including buffer overrun fixes, use puts
and putchar in places, etc (inspired by OpenBSD mods)
* disable progress bar when local-file is a pipe or '-'
* skip \r in http headers
* fix remote ftpd slash bug more elegantly (so it works with ////)
* abort_remote(): check if cout==NULL before using it. should fix [bin/3273]
* fixed up cosmetic problems when complete_remote() generated errors from the
remote server (such as "no files found", "login with user and pass", ...)
done by adding extra argument to remglob(), which is a pointer to an error
buffer to put messages in rather than printing to stdout.
 1.12 19-Jan-1997  lukem New features:
* Command line editing via editline(3) library.
* Context sensitive command and file completion, including remote files.

Enhancements to auto-fetch feature:
* Support for http:// URLs using the http protocol, including proxy HTTP
support via $htty_proxy if it's defined.
* The connection is kept open between successive files on the same host.
(obviously, this does not count for http requests.)
* Return value of ftp is 0 on no error, or the offset in argv[] of the file
which failed (i.e., argv[x] failed, ftp returns x).
* If the path in an ftp URL or classic format line has a trailing '/',
cd to the path and enter interactive mode. Fixes [bin/3011], albiet
requiring the user to help ftp in determining the operation.

Other changes:
* '-P port' works for normal ftp, and is the default for all classic style
auto-fetch transfers and for ftp URLs that don't specify the port.
(previously it would just work for the first xfer.)
* Some code moved into separated files along logical divisions.
* Editing and completion can be compiled out with -DSMALLFTP.
 1.11 09-Jan-1997  tls RCS ID police
 1.10 06-Dec-1996  lukem functionality mods:
* implement 'progress bar/meter' (inspired by ncftp). use 'progress' to
toggle on. it will display current file size to 5 digits, automatically
determining suffix (up to 16384 P (petabytes) == 2^64).
* 'ls' now uses NLST (unadorned listing), a la older ftp clients. 'dir'
still does LIST (long listing). idea from John Nemeth <jnemeth@cue.bc.ca>

bug fixes:
* return first line of reply in reply_string[] from getreply(), instead
of last line. This fixes [bin/741] (parsing of SYST), and also means
that SIZE and MDTM messages will be parsed correctly if they're longer
than 1 line.
* parse URL-style auto-ftps that have no filename correctly
(e.g, ftp://host, ftp://host/, ftp://host/dir/). pointed out by
Jaromir Dolecek <dolecek@saruman.ics.muni.cz>
* pass the correct size array in 2nd arg of utimes() when setting the
modification time
 1.9 28-Nov-1996  lukem More features, some of which were inspired by changes that
friedman@gnu.ai.mit.edu (Noah Friedman) made to his modified ftp:
- implement "lpwd" - local pwd
- implement "preserve" - toggle preserving of file modification
times on retrieved files
- allow for explicit "on" or "off" arg to toggle commands
- "exit" synonym for "quit", "msend" synonym for "mput"
- in confirmation mode, allow 'a' (yes to rest of current command),
and 'p' (turn off prompt mode, as if 'prompt off' was done,
effective immediately)
- "modtime" returns time formatted as localtime, not GMT

Bug fixes:
- check for extraneous args on commands
- cleanup const usage, line formatting
- create 0 length temporary file in remglob() to prevent symlink games
(from OpenBSD)
- check length of filename of ~/.netrc (from OpenBSD)
 1.8 25-Nov-1996  fvdl Correct the connecthelp info. This is really ftp, not tftp.
 1.7 25-Nov-1996  lukem cleanups/bugfixes:
- don't echo 'ACCT' parameter when debugging (a la 'PASS')
- Fix checking of directory access for "/foo", the parent
directory is "/", not "" (from FreeBSD)
- remove trailing whitespace on lines
- add any missing NetBSD tags
- cleanups to man page, includinging sorting options description

feature additions:
- variable sized hash marks (from [bin/683], but done in the hash command
as an optional arg)
- more user-friendly transfer time printing (from FreeBSD, with mods)
- '-p' command line option to jump into PASV mode (closes [bin/2857],
but with an option rather than checking argv[0])
- SIGINFO support for printing xfer stats when sending/receiving requests
- '-P port' for changing the port to connect to (from thorpej@netbsd.org)
- '-a': bypass normal login, and try anonymous login (from OpenBSD
via thorpej)
- autofetch files via url (ftp://...) or "classic" (host:/file)
(from OpenBSD via thorpej)
- 'ftp' synonymous with 'open' (from FreeBSD)
 1.6 08-Sep-1995  tls Sync with 4.4lite2
 1.5 29-Aug-1994  mycroft branches: 1.5.2;
Add RCS ids.
 1.4 25-Aug-1994  cgd passive mode support, from David Carrel <carrel@cisco.com>, and cleaned
up for the new ftp sources by me.
 1.3 25-Aug-1994  cgd clean up import, no local changes.
 1.2 01-Aug-1993  mycroft Add RCS identifiers.
 1.1 21-Mar-1993  cgd branches: 1.1.1; 1.1.2;
Initial revision
 1.1.2.1 25-Aug-1994  cgd import from 4.4-Lite
 1.1.1.2 05-Sep-1995  tls imported from 44lite2
 1.1.1.1 21-Mar-1993  cgd initial import of 386bsd-0.1 sources
 1.5.2.2 29-Aug-1994  mycroft Add RCS ids.
 1.5.2.1 29-Aug-1994  mycroft file cmdtab.c was added on branch netbsd-1-0 on 1994-08-29 03:09:09 +0000
 1.17.2.1 10-Nov-1998  cgd pull up rev(s) 1.18-1.19 from trunk. (feyrer)
 1.20.2.1 25-Jun-1999  cgd pull up rev 1.21 from trunk. (cgd)
 1.26.2.1 27-Dec-1999  wrstuden Pull up to last week's -current.
 1.43.2.2 24-Jul-2005  tron Pull up revision 1.45 (requested by lukem in ticket #606):
copyright maintenance
 1.43.2.1 09-May-2005  tron Pull up revision 1.44 (requested by lukem in ticket #266):
gratuitous whitespace cleanup (before someone else jumps the gun...)
 1.47.10.1 18-May-2008  yamt sync with head.
 1.48.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.50.6.1 13-May-2009  jym Sync with HEAD.

Third (and last) commit. See http://mail-index.netbsd.org/source-changes/2009/05/13/msg221222.html
 1.51.12.1 25-Feb-2013  tls resync with head
 1.51.8.1 17-Dec-2013  bouyer Apply patch, requested by tron in ticket #997:
usr.bin/ftp/Makefile patch
usr.bin/ftp/cmds.c patch
usr.bin/ftp/cmdtab.c patch
usr.bin/ftp/extern.h patch
usr.bin/ftp/fetch.c patch
usr.bin/ftp/ftp.1 patch
usr.bin/ftp/ftp.c patch
usr.bin/ftp/ftp_var.h patch
usr.bin/ftp/main.c patch
usr.bin/ftp/progressbar.c patch
usr.bin/ftp/ssl.c patch
usr.bin/ftp/ssl.h patch
usr.bin/ftp/util.c patch
usr.bin/ftp/version.h patch

Add HTTPS support to ftp(1).
 1.51.6.1 23-Jan-2013  yamt sync with head
 1.52.40.1 16-May-2023  martin Pull up following revision(s) (requested by lukem in ticket #170):

usr.bin/ftp/ssl.c: revision 1.13
usr.bin/ftp/fetch.c: revision 1.236
usr.bin/ftp/util.c: revision 1.166
usr.bin/ftp/main.c: revision 1.129
usr.bin/ftp/extern.h: revision 1.83
usr.bin/ftp/ftp.1: revision 1.148
usr.bin/ftp/cmdtab.c: revision 1.53
usr.bin/ftp/version.h: revision 1.96

Add option sslnoverify to control validation of SSL certificates.

Add netrc processing to fetch-mode (URL on command line) to enable options and autologin
via netrc.

Fix SSL cleanup in some error paths.

Certificate validation is now enabled by default. Set FTPSSLNOVERIFY=1 in environment
or configure a corresponding init macro via netrc to not validate certs (required if
you haven't installed a required CA certificate for OpenSSL).

Discussed with lukem@ on icb.
 1.48 25-Sep-2024  christos pass some lint.
 1.47 28-Jan-2019  christos branches: 1.47.10; 1.47.12;
PR/53916: Rob Gill: remove redundant assignment to matchlen.
 1.46 12-Apr-2009  lukem branches: 1.46.38; 1.46.46;
Fix numerous WARNS=4 issues (-Wcast-qual -Wsign-compare).
 1.45 12-Apr-2009  lukem fix -Wshadow issues
 1.44 30-Sep-2008  lukem branches: 1.44.6;
update copyrights
 1.43 28-Apr-2008  martin Remove clause 3 and 4 from TNF licenses
 1.42 17-Apr-2007  lukem branches: 1.42.10;
* Implement -s srcaddr; uses srcaddr as the local IP address for all
connections.
Based on code in the version of ftp that FreeBSD had before they
replaced it with lukemftp.
* Move error message handling into ftp_connect() rather than in the
caller, so that more specific error reporting can occur.
* Improve consistency of various warning and error messages.
 1.41 31-Jan-2006  christos Rename xfoo() to ftp_foo() to avoid collisions with libssh. Don't ask.
 1.40 09-Jun-2005  lukem copyright maintenance
 1.39 19-May-2005  lukem Some const cleanups.
 1.38 01-May-2000  lukem branches: 1.38.12;
convert to ANSI KNF
 1.37 20-Jan-2000  lukem complete_ambiguous(): be consistent about completing unambiguous matches; if
the word is already complete then return CC_REFRESH so that the higher layer
may append a suffix if necessary. Fix from Launey Thomas <ljt@alum.mit.edu>.
 1.36 28-Nov-1999  lukem * implement xsl_init() and xsl_add(); error checking forms of sl_{init,add}()
* fix bug where the second press of <TAB> on an empty word (i.e, list
all options) may have resulted in an strncmp() against NULL. (detected
by _DIAGASSERT())
 1.35 26-Nov-1999  lukem * complete_remote(): use remglob("", ...) instead of remglob(".", ...),
for listings of the current working directory; some ftp servers don't
like `NLST .'.
[noted by Giles Lean <giles@nemeton.com.au>]
* recvrequest(): treat remote=="" as remote==NULL when calling command().
(to support the above change)
* support `[user@]' in `[user@]host' and `[user@]host[:][path]'.
[based on idea (and initial code) from David Maxwell <david@fundy.ca>]
* `idle' may be invoked without any args
* reformat some comments
* reformat usage string in program and man page
* call updateremotepwd() after successful login, not after successful connect
* always call setsockopt(, IPPROTO_IP, IP_TOS, ) (et al); using #if
defined(IPPROTO_IP) doesn't work on certain foreign systems where
enums instead of #defines are used...
[noted by Matthias Pfaller <leo@dachau.marco.de>]
 1.34 24-Oct-1999  lukem new features:
- add `usage'; displays the usage of a command.
implemented by calling the c_handler() with argc = 0, argv = "funcname".
- add `passive auto'; does the same as $FTPMODE=auto.
- add `set [option value]'; display all options, or set an option to a value.
- add `unset option'; unset an option.
- add getoptionvalue() to retrieve an option's value, and replace a few
global variables with calls to this.
- implement cleanuppeer(), which resets various bits of state back to
`disconnected'. call in disconnect() and lostpeer().
- support completing on `options'.
- improve recovery after a SIGINT may have closed the connection.
XXX: there's still a couple to fix

other stuff:
- various consistency fixes in the man page.
- ensure that the command usage strings in the code and man page match reality.
- mput/mget: check that the connection still exists before each xfer.
- minor cosmetic changes in confirm().
- set code correctly in sizecmd() and modtime()
- don't need \n in err() strings.
- change lostpeer to take an argument (rather than casting (sig_t)lostpeer
in signal handlers)
- knf and whitespace police.
 1.33 05-Oct-1999  lukem add TNFi copyright to all files i've done more than a minor amount of work to...
 1.32 28-Sep-1999  lukem * add new commands:
lpage page local files
pdir as dir, but through your $PAGER
pls as ls, but through your $PAGER
* implement docase() (a la dotrans() et al) and use appropriately, rototilling
some duplicated code
* globulize(): modify to return a pointer to the strdup()ed result in all cases,
and hack the code that calls it to take this into account
* replace strcpy() and strncpy() with strlcpy()
* put(), getit(): use some aptly named local vars instead of argv[...]
* delint
 1.31 27-Sep-1999  lukem * idle(): rename to idlecmd(). certain linux distributions have an incompatible
prototype for idle() in <unistd.h> (which i thought was against namespace
and sensibility guidelines, but...)
* consistently use xsignal() instead of signal(). we get known behaviour
in all cases (SA_RESTART), which is good for some borken foreign systems.
* remove signal.h from most files; it's unnecessary now
* fetch_url(): use `long chunksize' instead of ssize_t; it's more portable, and
we're setting chunksize with strtol() anyway
* xsignal(): only use SA_RESTART if it exists. SunOS 4.x doesn't have it
but has the inverse (SA_INTERRUPT). the original function i was inspired
from had this support (lib/signal.c, W. Richard Stevens' `UNP 2nd ed Vol 1').
* remove <termios.h> from util.c; it should be unnecessary now
 1.30 26-Sep-1999  lukem * replace ifdefs against __SVR4 and __linux__ with DIRENT_MISSING_D_NAMLEN;
it's more portable and more obvious
* remove the mkgmtime() && HAVE_TIMEGM stuff:
a) why should netbsd have to define HAVE_TIMEGM to compile cleanly?
b) foreign compiles of ftp should just be linked with working
timegm function

a more portable version of this ftp client will be released as a 3rdparty
product; no use polluting our code with half-baked attempts...
 1.29 24-Sep-1999  lukem * use %lld instead of %qd to print out (long long) vars.
(slightly more portable; e.g, solaris supports this)
* remove some fluff (lint)
 1.28 24-Sep-1999  lukem in complete(), only copy the word if there is one.
`cd <TAB>' was triggering _DIAGASSERT(src != NULL)).
 1.27 22-Sep-1999  lukem branches: 1.27.2;
replace snprintf() with strlcpy(), strlcat(), or direct assignment
where appropriate. (strlc*() are easier to port to foriegn systems).

XXX; there's still a few snprintf's in the progress meter stuff to convert
 1.26 10-Jul-1999  christos don't do unnecessary const castaways
 1.25 24-Jun-1999  christos include <signal.h> explicitly! we use sig_t extern.h, and we cannot
depend on other headers fetching it on other OS's.
 1.24 20-Jun-1999  cgd split the "SMALL" #ifdefs into ones for NO_ABOUT, NO_EDITCOMPLETE, and
NO_PROGRESS. -DSMALL still implies all of those. progress meter support
isn't necessary for the smallest possible ftp client, but it adds very
little space and makes users' lives much better. Therefore, it should
be enabled for installation media if at all possible.
 1.23 12-Jun-1999  christos oops reversed the test.
 1.22 12-Jun-1999  christos Make this compile on linux [zoularis on linux works]
 1.21 08-Mar-1999  lukem branches: 1.21.2;
whitespace & copyright fixes
 1.20 18-Feb-1999  lukem during an ambiguous complete, take into account the length of the
string already input before inserting the non-ambiguous component.
bug noted by Thorsten Frueauf <frueauf@ira.uka.de> in [bin/7014]
 1.19 08-Feb-1999  lukem whitespace
 1.18 07-Feb-1999  lukem changes to fix [bin/6951] by Peter Simons <simons@cys.de>:
* implement ftpvis(), which \ quotes the following in the given string:
SP, TAB, \, CR, "
* use ftpvis() in complete_ambiguous(), to escape characters in a word
which would confuse makeargv()/slurpstring().
 1.17 07-Feb-1999  lukem complete ~ in local paths (noted by perry@netbsd.org)
 1.16 08-Aug-1998  lukem minor workaround for borken compilers
 1.15 03-Aug-1998  lukem features:
* support $no_proxy, which is a comma or space separated list of
host[:port] elements for which proxying is to be disabled.
(asked for by cgd in [bin/5027])
* if $FTPANONPASS is defined, use that as the anon ftp password
(instead of "`whoami`@")
* allow http URL's without a filename as long as an output file
is specified.

other stuff:
* implement parse_url(), which breaks up a URL into its bits, and use.
* simplify url_get() and auto_fetch() to use parse_url() and to not
modify the supplied URL or a copy of it.
* implement xmalloc() and xstrdup(); error-checked malloc()/strdup()
* add more consistency to messages, quoting strings in output as `%s'
 1.14 04-Jun-1998  lukem some fixes & enhancements from openbsd's ftp, with extra fixes by me:
* default to passive with active fallback. $FTPMODE modifies this behaviour.
-A forces active connection.
* support '-o outfile' for auto-fetched files. outfile can be a file,
`-' (for stdout), or '|command' (to output each file through command).
* support '-r waittime', which retries the connection after waittime seconds
if it fails.
* fix 'page file' when restart is non-zero.
* try all ip-addresses of a host in a http fetch (as the normal ftp fetch
does).

XXX: a ``broken pipe'' error sometimes occurs with -o '|command';
i haven't tracked this down yet.
 1.13 01-Jun-1998  lukem * use CC_REFRESH_BEEP in an ambiguous complete (recently added to -ledit)
* in command completion, append a space to a definite match
* in local path completion, append a space or / to a definite match,
depending if the path is a non-directory or directory respectively
(nothing added to remote path completion yet, until a simple sane method
of determining if the path is a directory or not is available).
 1.12 20-May-1998  christos - cast arg to is*() to unsigned char.
- don't require d_namlen to exist
- don't recompute the length of the file everytime around the loop.
 1.11 13-Sep-1997  lukem branches: 1.11.2;
Fixes from Todd Miller <Todd.Miller@courtesan.com>:
* use size_t instead of int in places
* use symbolic constants when using access()
 1.10 18-Aug-1997  lukem bugs fixed:
* don't interpret '-' or '|' when a local filename is determined from
the remote name (i.e, in mget, and in get with only one argument).
This is implemented using an extra argument to recvrequest().
Fixes a major security hole.
* clean up memory leak when using globulize()
* clean up a couple of comments
* fix wording in TNF copyright

features added:
* support for TIS fwtk gate-ftp servers:
* read defaults from $FTPSERVER && $FTPSERVERPORT
* start in gate-ftp mode if invoked as 'gate-ftp'
* toggle or set with 'gate [host [port]]'
 1.9 20-Jul-1997  lukem * use RCSID() && COPYRIGHT() macros
* cleanup for WARNS=1 (including some ugly '(void)&var' bits wrapped in
#ifdef __GNUC__ to shut up gcc warnings WRT setjmp/longjmp)
* use strtol() instead of atoi(), and more extensively check result of
conversion
* use u_int16_t instead of short or int for TCP port addresses
 1.8 24-May-1997  lukem Simplify NLST parsing code in complete_remote(); just basename each of
the returned names! Fixes parsing problems caused by the differences
between NLST output on BSD, Solaris, and Windows NT.
 1.7 14-Apr-1997  lukem More enhancements/bugfixes (when will it end?)
* differentiate between being connected, and being logged in
* cleanup some text messages
* support username & password ftp URLs (ftp://user:pass@host/) in non-proxy
situations; assume proxy supports it for proxy situations.
* cd to / before performing any autofetch transfers
* use strncasecmp in URL parsing. fix from <Todd.Miller@courtesan.com>
 1.6 16-Mar-1997  lukem Fixes from <Todd.Miller@cs.colorado.edu>, with some cleanup and reworks by me:
* only echo "Passive mode" in verbose mode; scripts that use ftp
may get unwanted output otherwise
* disable progress bar and modification time preservation when
retreiving to a non-regular files. fixes progress bar getting in
way of "get file /dev/tty"
* setup el_init() et al if editing is set, not if fromatty.
TODO: migrate this to a function, and call if editing is turned on later
in the session. also implement edit_cleanup if editing is turned off
* call el_set() after setting SIGWINCH handler. This fixes the problem
when suspending in a non-cbreak shell (e.g, csh) would trash your tty mode.
* reset interactive mode correctly in auto_fetch() mget mode
 1.5 14-Mar-1997  christos SMALLFTP->SMALL; remove SMALL from Makefile
 1.4 13-Mar-1997  christos Deal with SMALLFTP a bit differently [by compiling complete.c]
 1.3 13-Mar-1997  lukem Features:
* support remglobbing of auto_fetch arguments
* new flag - '-e'; disable editing
* "page file" == "get file |${PAGER-less}"

Bugfixes/cleanup:
* consistently use a trailing '.' on messages
* code cleanup, including buffer overrun fixes, use puts
and putchar in places, etc (inspired by OpenBSD mods)
* disable progress bar when local-file is a pipe or '-'
* skip \r in http headers
* fix remote ftpd slash bug more elegantly (so it works with ////)
* abort_remote(): check if cout==NULL before using it. should fix [bin/3273]
* fixed up cosmetic problems when complete_remote() generated errors from the
remote server (such as "no files found", "login with user and pass", ...)
done by adding extra argument to remglob(), which is a pointer to an error
buffer to put messages in rather than printing to stdout.
 1.2 01-Feb-1997  lukem [Yet Another Huge Ftp Commit - hopefully the last for a while,
barring any more little things people want added ...]

New features:
* progressmeter is now asynchronous, so "stalled" transfers can be
detected. "- stalled -" is displayed instead of the ETA in this case.
When the xfer resumes, the time that the xfer was stalled for is
factored out of the ETA. It is debatable whether this is better than
not factoring it out, but I like it this way (I.e, if it stalls for 8
seconds and the ETA was 30 seconds, when it resumes the ETA will still
be 30 seconds).
* verbosity can be disabled on the command line (-V), so that in auto-fetch
mode the only lines displayed will be a description of the file, and
the progress bar (if possible)
* if the screen is resized (and detected via the SIGWINCH signal), the
progress bar will rescale automatically.

Bugs fixed:
* progress bar will not use the last character on the line, as this can
cause problems on some terminals
* screen dimensions (via ioctl(TIOCWINSZ)) should use stdout not stdin
* progressmeter() used some vars before initialising them
* ^D will quit now. [fixes bin/3162]
* use hstrerror() to generate error message for host name lookup failure.
* use getcwd instead of getwd (it should have been OK, but why tempt fate?)
* auto-fetch transfers will always return a positive exit value upon failure
or interruption, relative to the file's position in argv[].
* remote completion of / will work, without putting a leading "///".
This is actually a bug in ftpd(1), where "NLST /" prefixes all names
with "//", but fixing every ftpd(1) is not an option...
 1.1 19-Jan-1997  lukem New features:
* Command line editing via editline(3) library.
* Context sensitive command and file completion, including remote files.

Enhancements to auto-fetch feature:
* Support for http:// URLs using the http protocol, including proxy HTTP
support via $htty_proxy if it's defined.
* The connection is kept open between successive files on the same host.
(obviously, this does not count for http requests.)
* Return value of ftp is 0 on no error, or the offset in argv[] of the file
which failed (i.e., argv[x] failed, ftp returns x).
* If the path in an ftp URL or classic format line has a trailing '/',
cd to the path and enter interactive mode. Fixes [bin/3011], albiet
requiring the user to help ftp in determining the operation.

Other changes:
* '-P port' works for normal ftp, and is the default for all classic style
auto-fetch transfers and for ftp URLs that don't specify the port.
(previously it would just work for the first xfer.)
* Some code moved into separated files along logical divisions.
* Editing and completion can be compiled out with -DSMALLFTP.
 1.11.2.1 10-Nov-1998  cgd pull up rev(s) 1.12-1.16 from trunk and patch to work on 1.3 branch. (feyrer)
 1.21.2.1 25-Jun-1999  cgd pull up revs 1.22-1.24 from trunk. (cgd)
 1.27.2.1 27-Dec-1999  wrstuden Pull up to last week's -current.
 1.38.12.2 24-Jul-2005  tron Pull up revision 1.40 (requested by lukem in ticket #606):
copyright maintenance
 1.38.12.1 24-Jul-2005  tron Pull up revision 1.39 (requested by lukem in ticket #606):
Some const cleanups.
 1.42.10.1 18-May-2008  yamt sync with head.
 1.44.6.1 13-May-2009  jym Sync with HEAD.

Third (and last) commit. See http://mail-index.netbsd.org/source-changes/2009/05/13/msg221222.html
 1.46.46.1 10-Jun-2019  christos Sync with HEAD
 1.46.38.3 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1763:

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/cmds.c up to 1.141
usr.bin/ftp/complete.c up to 1.47
usr.bin/ftp/domacro.c up to 1.23
usr.bin/ftp/extern.h up to 1.82
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp.c up to 1.174
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/progressbar.c up to 1.24
usr.bin/ftp/progressbar.h up to 1.9
usr.bin/ftp/ssl.c up to 1.12
usr.bin/ftp/ssl.h up to 1.5
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

ftp(1): validate address from PASV and LPSV response.
ftp(1): use raw write(2) instead of fwrite(3) to avoid stream
corruption because of the progress bar interrupts.
Fixes for PR 56219 and PR 55857.
PR 57003: Support relative redirects.
 1.46.38.2 12-Sep-2022  martin Backout ticket #1763 for now - trust anchors are not solved.
 1.46.38.1 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1763:

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/cmds.c up to 1.141
usr.bin/ftp/complete.c up to 1.47
usr.bin/ftp/domacro.c up to 1.23
usr.bin/ftp/extern.h up to 1.82
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp.c up to 1.174
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/progressbar.c up to 1.24
usr.bin/ftp/progressbar.h up to 1.9
usr.bin/ftp/ssl.c up to 1.11
usr.bin/ftp/ssl.h up to 1.5
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

ftp(1): validate address from PASV and LPSV response.
ftp(1): use raw write(2) instead of fwrite(3) to avoid stream
corruption because of the progress bar interrupts.
Fixes for PR 56219 and PR 55857.
PR 57003: Support relative redirects.
 1.47.12.1 02-Aug-2025  perseant Sync with HEAD
 1.47.10.1 13-Oct-2024  martin Pull up following revision(s) (requested by riastradh in ticket #970):

tests/usr.bin/Makefile: revision 1.42
usr.bin/ftp/ruserpass.c: revision 1.34
usr.bin/ftp/main.c: revision 1.130
usr.bin/ftp/ssl.c: revision 1.17
usr.bin/ftp/main.c: revision 1.131
usr.bin/ftp/ssl.c: revision 1.18
usr.bin/ftp/main.c: revision 1.132
usr.bin/ftp/ssl.c: revision 1.19
usr.bin/ftp/main.c: revision 1.133
distrib/sets/lists/tests/mi: revision 1.1342
usr.bin/ftp/ftp.1: revision 1.151
usr.bin/ftp/ftp.1: revision 1.152
usr.bin/ftp/progressbar.c: revision 1.25
usr.bin/ftp/ftp.1: revision 1.153
usr.bin/ftp/progressbar.c: revision 1.26
usr.bin/ftp/ftp.1: revision 1.155
usr.bin/ftp/ftp.1: revision 1.156
usr.bin/ftp/fetch.c: revision 1.239
usr.bin/ftp/ftp.1: revision 1.157
usr.bin/ftp/ftp.1: revision 1.158
usr.bin/ftp/ftp.1: revision 1.159
usr.bin/ftp/ftp_var.h: revision 1.87
etc/mtree/NetBSD.dist.tests: revision 1.208
usr.bin/ftp/ftp_var.h: revision 1.88
usr.bin/ftp/ftp_var.h: revision 1.89
usr.bin/ftp/cmds.c: revision 1.142
usr.bin/ftp/util.c: revision 1.168
usr.bin/ftp/cmds.c: revision 1.143
tests/usr.bin/ftp/custom_headers.sh: revision 1.1
usr.bin/ftp/ssl.c: revision 1.20
usr.bin/ftp/complete.c: revision 1.48
tests/usr.bin/ftp/Makefile: revision 1.1
tests/usr.bin/ftp/t_custom_headers.sh: revision 1.1
usr.bin/ftp/fetch.c: revision 1.240
usr.bin/ftp/fetch.c: revision 1.241
usr.bin/ftp/ftp.c: revision 1.176
usr.bin/ftp/ftp.c: revision 1.177
(all via patch)

ftp(1): wording and formatting improvements

Fix grammar issue with "Support values" reported in private mail.
Document all file transfer types in "type" and cross-reference that.
Consistency fixes in describing file transfer parameters and types.

Fix some mandoc -Tlint issues (except "useless macro: Tn").

Add -b <buflen> to specify the buffer size.

ftp: bump FTPBUFLEN from 4kB to 16kB
sourceforge.net returns a 5kB content-security-policy.
Analyzed by mlelstv@ who reports usual limits are between 4kB and 48kB.
default is now 16K

ftp: improve -b documentation

Order -b bufsize in the synopsis.

Document the actual default value.

ftp: improve units used in comments and errors
Use "KiB" instead of "K" in errors.
Clarify related comments.

pass some lint.

PR/58581: Sunil Nimmagadda: Add flag to allow specifying extra http header
fields.

ftp(1): Nix trailing whitespace in man page.
No functional change intended.

PR bin/58581: ftp(1) should allow specifying header fields in http requests
fix markup (h -> H), explain about multiple headers, fix usage (from RVP)

Don't forget the dot, use the intended macro name (I think),
and improve the wording a little. (All related to the -H option.)
ftp(1): Add test for custom HTTP header fields.

Based on a patch from Sunil Nimmagadda.

PR bin/58581: ftp(1) should allow specifying header fields in http
requests
 1.23 03-Feb-2019  mrg - add or adjust /* FALLTHROUGH */ where appropriate
- add __unreachable() after functions that can return but won't in
this case, and thus can't be marked __dead easily
 1.22 12-Apr-2009  lukem branches: 1.22.38; 1.22.46;
Fix numerous WARNS=4 issues (-Wcast-qual -Wsign-compare).
 1.21 29-Jun-2005  christos branches: 1.21.30;
Add NO_USAGE and NO_DEBUG so that we can fit in the floppies again.
 1.20 07-Aug-2003  agc branches: 1.20.6;
Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22365, verified by myself.
 1.19 01-Feb-2002  itojun line2 may overrun if line is too long (> 200). be more careful on strcpy.
XXX strlen(argv[x]) should be checked before copies.
 1.18 15-Dec-2000  lukem invoke cmdtab.c_handler()s with argv[0] == c_name instead of the
supplied name. that way the full (unambiguous) name is displayed in
error messages and usage strings.
 1.17 18-Jul-2000  lukem convert to new knf
 1.16 01-May-2000  lukem convert to ANSI KNF
 1.15 11-Nov-1999  lukem whitespace nits
 1.14 24-Oct-1999  lukem new features:
- add `usage'; displays the usage of a command.
implemented by calling the c_handler() with argc = 0, argv = "funcname".
- add `passive auto'; does the same as $FTPMODE=auto.
- add `set [option value]'; display all options, or set an option to a value.
- add `unset option'; unset an option.
- add getoptionvalue() to retrieve an option's value, and replace a few
global variables with calls to this.
- implement cleanuppeer(), which resets various bits of state back to
`disconnected'. call in disconnect() and lostpeer().
- support completing on `options'.
- improve recovery after a SIGINT may have closed the connection.
XXX: there's still a couple to fix

other stuff:
- various consistency fixes in the man page.
- ensure that the command usage strings in the code and man page match reality.
- mput/mget: check that the connection still exists before each xfer.
- minor cosmetic changes in confirm().
- set code correctly in sizecmd() and modtime()
- don't need \n in err() strings.
- change lostpeer to take an argument (rather than casting (sig_t)lostpeer
in signal handlers)
- knf and whitespace police.
 1.13 27-Sep-1999  lukem * idle(): rename to idlecmd(). certain linux distributions have an incompatible
prototype for idle() in <unistd.h> (which i thought was against namespace
and sensibility guidelines, but...)
* consistently use xsignal() instead of signal(). we get known behaviour
in all cases (SA_RESTART), which is good for some borken foreign systems.
* remove signal.h from most files; it's unnecessary now
* fetch_url(): use `long chunksize' instead of ssize_t; it's more portable, and
we're setting chunksize with strtol() anyway
* xsignal(): only use SA_RESTART if it exists. SunOS 4.x doesn't have it
but has the inverse (SA_INTERRUPT). the original function i was inspired
from had this support (lib/signal.c, W. Richard Stevens' `UNP 2nd ed Vol 1').
* remove <termios.h> from util.c; it should be unnecessary now
 1.12 04-Jun-1998  lukem branches: 1.12.4;
some fixes & enhancements from openbsd's ftp, with extra fixes by me:
* default to passive with active fallback. $FTPMODE modifies this behaviour.
-A forces active connection.
* support '-o outfile' for auto-fetched files. outfile can be a file,
`-' (for stdout), or '|command' (to output each file through command).
* support '-r waittime', which retries the connection after waittime seconds
if it fails.
* fix 'page file' when restart is non-zero.
* try all ip-addresses of a host in a http fetch (as the normal ftp fetch
does).

XXX: a ``broken pipe'' error sometimes occurs with -o '|command';
i haven't tracked this down yet.
 1.11 20-May-1998  christos - cast arg to is*() to unsigned char.
 1.10 20-Jul-1997  lukem branches: 1.10.2;
* use RCSID() && COPYRIGHT() macros
* cleanup for WARNS=1 (including some ugly '(void)&var' bits wrapped in
#ifdef __GNUC__ to shut up gcc warnings WRT setjmp/longjmp)
* use strtol() instead of atoi(), and more extensively check result of
conversion
* use u_int16_t instead of short or int for TCP port addresses
 1.9 13-Mar-1997  lukem Features:
* support remglobbing of auto_fetch arguments
* new flag - '-e'; disable editing
* "page file" == "get file |${PAGER-less}"

Bugfixes/cleanup:
* consistently use a trailing '.' on messages
* code cleanup, including buffer overrun fixes, use puts
and putchar in places, etc (inspired by OpenBSD mods)
* disable progress bar when local-file is a pipe or '-'
* skip \r in http headers
* fix remote ftpd slash bug more elegantly (so it works with ////)
* abort_remote(): check if cout==NULL before using it. should fix [bin/3273]
* fixed up cosmetic problems when complete_remote() generated errors from the
remote server (such as "no files found", "login with user and pass", ...)
done by adding extra argument to remglob(), which is a pointer to an error
buffer to put messages in rather than printing to stdout.
 1.8 19-Jan-1997  lukem New features:
* Command line editing via editline(3) library.
* Context sensitive command and file completion, including remote files.

Enhancements to auto-fetch feature:
* Support for http:// URLs using the http protocol, including proxy HTTP
support via $htty_proxy if it's defined.
* The connection is kept open between successive files on the same host.
(obviously, this does not count for http requests.)
* Return value of ftp is 0 on no error, or the offset in argv[] of the file
which failed (i.e., argv[x] failed, ftp returns x).
* If the path in an ftp URL or classic format line has a trailing '/',
cd to the path and enter interactive mode. Fixes [bin/3011], albiet
requiring the user to help ftp in determining the operation.

Other changes:
* '-P port' works for normal ftp, and is the default for all classic style
auto-fetch transfers and for ftp URLs that don't specify the port.
(previously it would just work for the first xfer.)
* Some code moved into separated files along logical divisions.
* Editing and completion can be compiled out with -DSMALLFTP.
 1.7 09-Jan-1997  tls RCS ID police
 1.6 28-Nov-1996  lukem More features, some of which were inspired by changes that
friedman@gnu.ai.mit.edu (Noah Friedman) made to his modified ftp:
- implement "lpwd" - local pwd
- implement "preserve" - toggle preserving of file modification
times on retrieved files
- allow for explicit "on" or "off" arg to toggle commands
- "exit" synonym for "quit", "msend" synonym for "mput"
- in confirmation mode, allow 'a' (yes to rest of current command),
and 'p' (turn off prompt mode, as if 'prompt off' was done,
effective immediately)
- "modtime" returns time formatted as localtime, not GMT

Bug fixes:
- check for extraneous args on commands
- cleanup const usage, line formatting
- create 0 length temporary file in remglob() to prevent symlink games
(from OpenBSD)
- check length of filename of ~/.netrc (from OpenBSD)
 1.5 08-Sep-1995  tls Sync with 4.4lite2
 1.4 29-Aug-1994  mycroft branches: 1.4.2;
Add RCS ids.
 1.3 25-Aug-1994  cgd clean up import, no local changes.
 1.2 01-Aug-1993  mycroft Add RCS identifiers.
 1.1 21-Mar-1993  cgd branches: 1.1.1; 1.1.2;
Initial revision
 1.1.2.1 25-Aug-1994  cgd import from 4.4-Lite
 1.1.1.2 05-Sep-1995  tls imported from 44lite2
 1.1.1.1 21-Mar-1993  cgd initial import of 386bsd-0.1 sources
 1.4.2.2 29-Aug-1994  mycroft Add RCS ids.
 1.4.2.1 29-Aug-1994  mycroft file domacro.c was added on branch netbsd-1-0 on 1994-08-29 03:09:10 +0000
 1.10.2.1 10-Nov-1998  cgd pull up rev(s) 1.11-1.12 from trunk. (feyrer)
 1.12.4.1 27-Dec-1999  wrstuden Pull up to last week's -current.
 1.20.6.1 24-Jul-2005  tron Pull up revision 1.21 (requested by lukem in ticket #606):
Add NO_USAGE and NO_DEBUG so that we can fit in the floppies again.
 1.21.30.1 13-May-2009  jym Sync with HEAD.

Third (and last) commit. See http://mail-index.netbsd.org/source-changes/2009/05/13/msg221222.html
 1.22.46.1 10-Jun-2019  christos Sync with HEAD
 1.22.38.3 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1763:

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/cmds.c up to 1.141
usr.bin/ftp/complete.c up to 1.47
usr.bin/ftp/domacro.c up to 1.23
usr.bin/ftp/extern.h up to 1.82
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp.c up to 1.174
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/progressbar.c up to 1.24
usr.bin/ftp/progressbar.h up to 1.9
usr.bin/ftp/ssl.c up to 1.12
usr.bin/ftp/ssl.h up to 1.5
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

ftp(1): validate address from PASV and LPSV response.
ftp(1): use raw write(2) instead of fwrite(3) to avoid stream
corruption because of the progress bar interrupts.
Fixes for PR 56219 and PR 55857.
PR 57003: Support relative redirects.
 1.22.38.2 12-Sep-2022  martin Backout ticket #1763 for now - trust anchors are not solved.
 1.22.38.1 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1763:

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/cmds.c up to 1.141
usr.bin/ftp/complete.c up to 1.47
usr.bin/ftp/domacro.c up to 1.23
usr.bin/ftp/extern.h up to 1.82
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp.c up to 1.174
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/progressbar.c up to 1.24
usr.bin/ftp/progressbar.h up to 1.9
usr.bin/ftp/ssl.c up to 1.11
usr.bin/ftp/ssl.h up to 1.5
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

ftp(1): validate address from PASV and LPSV response.
ftp(1): use raw write(2) instead of fwrite(3) to avoid stream
corruption because of the progress bar interrupts.
Fixes for PR 56219 and PR 55857.
PR 57003: Support relative redirects.
 1.84 04-Oct-2024  christos remove const when string gets overwritten.
 1.83 25-Feb-2023  mlelstv branches: 1.83.2;
Add option sslnoverify to control validation of SSL certificates.
Add netrc processing to fetch-mode (URL on command line) to enable options and autologin
via netrc.
Fix SSL cleanup in some error paths.

Certificate validation is now enabled by default. Set FTPSSLNOVERIFY=1 in environment
or configure a corresponding init macro via netrc to not validate certs (required if
you haven't installed a required CA certificate for OpenSSL).

Discussed with lukem@ on icb.
 1.82 22-Jun-2019  christos branches: 1.82.10;
trim down error checking if we are small.
 1.81 04-Feb-2019  mrg - add justquit() that always exits. use it to avoid unreachable code.
 1.80 04-Jul-2012  is branches: 1.80.24; 1.80.32;
As discussed on tech-net@: Don't display expected EHOSTUNREACH for all but
the last connect attempts in terse mode.
 1.79 16-Sep-2011  joerg branches: 1.79.2; 1.79.4;
Use __dead
 1.78 04-Mar-2010  lukem Parse HTTP 'Date' entries in the `C' locale rather than the user's.
Fix from [bin/42917] (with minor changes), from KAMADA Ken'ichi.
 1.77 13-Jul-2009  roy Rename internal getline() function to get_line() so it does
conflict with the soon to be added getline(3) libc function.
 1.76 12-Apr-2009  lukem Fix numerous WARNS=4 issues (-Wcast-qual -Wsign-compare).
 1.75 10-May-2008  skd branches: 1.75.4; 1.75.6;
Add epsv6 and epsv to disable extended passive mode for ipv6 or both ipv4 and ipv6 respectively. This hack is due to our friends a Juniper Networks who break
epsv in ipv6. Should be fixed in ScreenOS 6.2.X.
 1.74 28-Apr-2008  martin branches: 1.74.2;
Remove clause 3 and 4 from TNF licenses
 1.73 22-Apr-2008  lukem Use the service name to getaddrinfo() (along with the host name), so that
features such as DNS Service Discovery have a better chance of working.
Suggested by David Young <dyoung>.

Display the service name in various status & error messages.

Don't getservbyname() the :port component of a URL; RFC 3986 says it's
just an unsigned number, not a service name.
 1.72 24-May-2007  lukem branches: 1.72.10;
Display times in RFC2822 form rather than using ctime(3), since
the former is more explicit about the timezone offset.
 1.71 18-Apr-2007  lukem Clean up use of confirm() and mbort() so that the current operation
is always passed in (instead of depending upon the 'mname' global).
For confirm(), if the second argument is NULL print the "Continue with <cmd>"
prompt. This fixes up the the display of interrupted prompts.
 1.70 31-Jan-2006  christos Rename xfoo() to ftp_foo() to avoid collisions with libssh. Don't ask.
 1.69 10-Jun-2005  lukem Implement:
int getline(FILE *stream, char *buf, size_t buflen, const char **errormsg)
Read a line from the FILE stream into buf/buflen using fgets(), so up
to buflen-1 chars will be read and the result will be NUL terminated.
If the line has a trailing newline it will be removed.
If the line is too long, excess characters will be read until
newline/EOF/error.
Various -ve return values indicate different errors, and errormsg
will be changed to an error description if it's not NULL.

Convert to use getline() instead of fgets() whenever reading user input
to ensure that an overly long input line doesn't leave excess characters
for the next input operation to accidentally use as input.

Zero out the password & account after we've finished with it.

Consistently use getpass(3) (i.e, character echo suppressed) when
reading the account data. For some reason, historically the "login"
code suppressed echo for Account: yet the "user" command did not!

Display the hostname in the "getaddrinfo failed" warning.

Appease some -Wcast-qual warnings. Fixing all of these requires
significant code refactoring. (mmm, legacy code).
 1.68 19-May-2005  lukem Some const cleanups.
 1.67 14-May-2005  lukem Fix some cast issues highlighted by Scott Reynolds using gcc 4 on OSX.4
 1.66 11-Apr-2005  lukem gratuitous whitespace cleanup (before someone else jumps the gun...)
 1.65 11-Apr-2005  lukem Implement a timeout on the accept(2) in dataconn() and the
connect(2) in xconnect() by temporarily setting O_NONBLOCK
on the socket and using xpoll() to wait for the operation
to succeed.
The timeout used is the '-q quittime' argument (defaults to
60s for accept(2), and the system default for connect(2)).
Idea inspired by discussion with Chuck Cranor.
This may (indirectly) fix various problems with timeouts
in active mode through broken firewalls.

Implement xpoll() as a wrapper around poll(2), to make it
easier to replace on systems without a functional poll(2).
Unconditionally use xpoll() instead of conditionally using
select(2) or poll(2).
 1.64 09-Feb-2005  christos branches: 1.64.2;
Factor out common string processing code eliminating static buffers,
making functions that should be static be static, and cleaning up
const usage. Added a guard against buffer overflow, but the domap function
is a bit too complicated for me to tackle right now. I will leave it
to the author; hi luke!
 1.63 03-Jan-2005  lukem Forbid filenames returned from mget that aren't in (or below) the
current directory.
The previous behaviour (of trusting the remote server's response when
retrieving the list of files to mget with prompting disabled) has been
in ftp ~forever, and has been a "known issue" for a long time.
Recently an advisory was published by D.J. Bernstein on behalf of
Yosef Klein warning of the problems with the previous behaviour, so
to alleviate concern I've fixed this with a sledgehammer.

Remember the local cwd after any operation which may change it.
Use "remotecwd" instead of "remotepwd".
 1.62 07-Aug-2003  agc Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22365, verified by myself.
 1.61 21-Jan-2003  jhawk Abstract out progress bar support into progressbar.[ch], for inclusion
in external programs (conditionalized on -DSTANDALONE_PROGRESS).

The following moved from util.c to progressbar.c:
alarmtimer(), progressmeter(), psummary(), ptransfer(),
xsignal(), xsignal_restart()

The following moved from extern.h and ftp_var.h to progressbar.h:
STALLTIME, verbose, fromatty, progress, quit_time, ttywidth
 1.60 11-Oct-2000  is branches: 1.60.2;
More format string cleanup by sommerfeld.
 1.59 06-Aug-2000  lukem * implement parseport(), which takes a string and attempts to convert
it to a numeric port number
* use parseport() in parse_url() and hookup()
* don't try and lookup the port number using getaddrinfo(), as it's too hard
to separate a failed host name lookup from a failed service name lookup.
this was causing lossage on systems that don't have `http' in services(5)
(such as solaris), but only crept in when we started using getaddrinfo()
unconditionally.
 1.58 18-Jul-2000  lukem add support for FEAT and OPTS commands with `features' and `opts'.
(from RFC 2389).

add support for MLST & MLSD (machine parseble listings) with 'mlst', 'mlsd'
and 'pmlsd' (mlsd |$PAGER) commands. (from draft-ietf-ftpext-mlst-11)

rename remotesyst() to getremoteinfo(), and modify to parse the result from
FEAT (if supported), and take into account the support for the various
extensions such as MDTM, SIZE, REST (STREAM), MLSD, and FEAT/OPTS.
put each feature into one of the following categories:
- known to work (explicit FEAT)
- unknown but assume works until explicit failure, when it's
then tagged as `known not to work'.
- known not to work (FEAT succeeded but didn't return anything,
or was unknown and then explicit failure)
assign results into features[] matrix.

add support to getreply() so that an optional callback will be called
for each line received from the server except for the first and last.
this is used in FEAT (and MLST) parsing.

modify various commands to check if REST (STREAM), MDTM and SIZE are
explicitly or implicitly supported before using.

fix `syst' when verbose is off.

minor knf (indent goto labels by one space, etc).

simply various command usage handlers by assuming that argv != NULL except
for quit() and disconnect().
 1.57 15-Jun-2000  lukem branches: 1.57.2;
* migrate the SYST parsing from setpeer() into a separate remotesyst().
call remotesyst() only when login has been successful
some servers don't let you run SYST until you've successfully logged in.
* in fetch_ftp(), always call setpeer() with autologin disabled, and use
the following ftp_login() to DTRT.
this prevents ftp from trying to login a second time if the first autologin
fails when connecting to a remote site anonymously using autofetch.
* reset unix_proxy and unix_server in cleanuppeer()
* missed a function conversion in the KNF sweep...
 1.56 31-May-2000  lukem Add support for 'ftp -u url file ...', to upload a list of files to given url.
Mostly based on [bin/10019] by Scott Aaron Bamford <sab@ansic.net>
 1.55 29-May-2000  itojun convert IPv4 mapped address (::ffff:10.1.1.1) into real IPv4 address
before touching it. IPv4 mapped address complicates too many things
in FTP protocol handling.
 1.54 01-May-2000  lukem branches: 1.54.2;
convert to ANSI KNF
 1.53 01-May-2000  lukem * Add support for `fget localfile', which reads a list of filenames to
retrieve from localfile. Based on work by Darren Reed.
* Crank version.
* Update copyright dates.
 1.52 31-Jan-2000  lukem define private type `sigfunc' as
typedef void (*sigfunc) __P((int));
and replace use of sig_t and void (*)(int).

certain other OSes define sig_t differently to that (they add extra arguments),
and it causes problems due to function mismatches, etc...
 1.51 28-Nov-1999  lukem * implement xsl_init() and xsl_add(); error checking forms of sl_{init,add}()
* fix bug where the second press of <TAB> on an empty word (i.e, list
all options) may have resulted in an strncmp() against NULL. (detected
by _DIAGASSERT())
 1.50 12-Nov-1999  lukem - implement `set rprompt'; right side version of `set prompt'. depends on
EL_RPROMPT support i added to editline(3).
- allow $FTPPROMPT and $FTPRPROMPT to override defaults for the relevant
prompts
- move `%' formatting code from prompt() to expandbuf().
- implement `%.' and `%c', similar to the same % codes in tcsh(1)
(functionality I added to tcsh nearly 6 years ago), except that `%.'
always does `...trailing' and `%c' always does `/<x>trailing'.
- unknown `%foo' codes get printed as `%foo'
 1.49 11-Nov-1999  lukem - implement updateremotepwd(); update the global variable `remotepwd'
to contain the remote working directory.
- add `set prompt', a user configurable prompt. (defaults to `ftp> ').
the following escape characters a la tcsh(1) are supported: %/, %m,
%M, and %n.
- add global var `username'; used by prompt code
- fix a couple of minor memory leaks
- bump version
 1.48 24-Oct-1999  lukem new features:
- add `usage'; displays the usage of a command.
implemented by calling the c_handler() with argc = 0, argv = "funcname".
- add `passive auto'; does the same as $FTPMODE=auto.
- add `set [option value]'; display all options, or set an option to a value.
- add `unset option'; unset an option.
- add getoptionvalue() to retrieve an option's value, and replace a few
global variables with calls to this.
- implement cleanuppeer(), which resets various bits of state back to
`disconnected'. call in disconnect() and lostpeer().
- support completing on `options'.
- improve recovery after a SIGINT may have closed the connection.
XXX: there's still a couple to fix

other stuff:
- various consistency fixes in the man page.
- ensure that the command usage strings in the code and man page match reality.
- mput/mget: check that the connection still exists before each xfer.
- minor cosmetic changes in confirm().
- set code correctly in sizecmd() and modtime()
- don't need \n in err() strings.
- change lostpeer to take an argument (rather than casting (sig_t)lostpeer
in signal handlers)
- knf and whitespace police.
 1.47 13-Oct-1999  lukem * fix up confirm() (broke `a' and `p' in last commit)
* simplify main loop (don't need `top' variable any more)
* use a struct sockaddr_in6.sin6_addr for the result from inet_pton(),
rather than u_char buf[16]
* add a few more comments
 1.46 12-Oct-1999  lukem a few user interface and cosmetic tweaks:
* confirm(): move from util.c to cmds.c. display mnemonic string in its prompt.
add support for `q' (terminate current xfer), `?' (show help list)
* in various signal handlers, output a linefeed only if fromatty.
* if fgets(stdin) returned NULL (i.e, EOF), clearerr(stdin) because you don't
want future fgets to fail. this is not done for the fgets() in the main
command loop, since ftp will quit at that point.
* unless ftp is invoked with -a, don't retain the anonftp setting between
hosts (`ftp somehost:' sets anonftp, but you don't want that to `stick'
if you close that connection and open a new one).
 1.45 09-Oct-1999  lukem allow a second SIGINT during the "xfer aborted. waiting for remote to finish abort."
stage. if this occurs, just call lostpeer() to close the connection. whilst this
might be considered brutal, it's also extremely handy if you're impatient or there's
lossage at the remote end.
 1.44 09-Oct-1999  lukem * use sigsetjmp()/siglongjump() instead of setjmp()/longjmp(); the latter
don't save the signal mask on some foreign systems.
* ensure signal handlers don't use stdio and do reset errno if they
don't exit with siglongjmp()
* use a common SIGINT handler for {send,recv}request()
 1.43 05-Oct-1999  lukem add TNFi copyright to all files i've done more than a minor amount of work to...
 1.42 05-Oct-1999  lukem enhancments from Marc Horowitz <marc@mit.edu> to improve connection timeouts:
* implement xsignal_restart(), which only sets the SA_RESTART flag if
specifically requested
* xsignal() is now a wrapper to xsignal_restart(). INFO, USR1, USR2 and WINCH
are restartable, ALRM, INT, PIPE and QUIT are not.
* improve getreply()'s timeout code to take advantage of the above.

other changes:
* improve wording of how globbing works for `classic' URLs (host:path).
suggested by John Refling <johnr@imageworks.com> in relation to PRs
[bin/8519] and [bin/8520]
* always compile in the `edit' command even if NO_EDITCOMPLETE defined.
it's just a no-op in the latter case, which is more consistent to
the users.
* always compile in about: support (i.e, remove NO_ABOUT).
i'm entitled to some vanity in this program...
* clean up some whitespace
 1.41 01-Oct-1999  lukem prefix the global variables in ftp_var.h with GLOBAL, which defaults
to "extern" if it's not set. define GLOBAL to (empty) in main.c.
this effectively moves all the globals into main.c whilst retaining
namespace access to them in other source files.
(global vars in header files confuse foreign linkers)
 1.40 30-Sep-1999  lukem * In the !NI_NUMERICHOST case (i.e, getaddrinfo() challenged systems), portnum
should be in host order. found/fixed by Matthias Pfaller <leo@dachau.marco.de>
* parse_url(): improve checking of portnum, and add an extra argument to pass
back the parsed portnum to the caller (reduces a bit of code duplication)
* Move the KAME/WIDE copyrights after the BSD/TNFi ones. Since there was
significantly less code added under the former, it's only fair on the latter.
 1.39 30-Sep-1999  lukem * fix initialisation of home[]
* fetch_url(): if path would be NULL, return strdup("")
 1.38 28-Sep-1999  lukem * add new commands:
lpage page local files
pdir as dir, but through your $PAGER
pls as ls, but through your $PAGER
* implement docase() (a la dotrans() et al) and use appropriately, rototilling
some duplicated code
* globulize(): modify to return a pointer to the strdup()ed result in all cases,
and hack the code that calls it to take this into account
* replace strcpy() and strncpy() with strlcpy()
* put(), getit(): use some aptly named local vars instead of argv[...]
* delint
 1.37 27-Sep-1999  lukem * idle(): rename to idlecmd(). certain linux distributions have an incompatible
prototype for idle() in <unistd.h> (which i thought was against namespace
and sensibility guidelines, but...)
* consistently use xsignal() instead of signal(). we get known behaviour
in all cases (SA_RESTART), which is good for some borken foreign systems.
* remove signal.h from most files; it's unnecessary now
* fetch_url(): use `long chunksize' instead of ssize_t; it's more portable, and
we're setting chunksize with strtol() anyway
* xsignal(): only use SA_RESTART if it exists. SunOS 4.x doesn't have it
but has the inverse (SA_INTERRUPT). the original function i was inspired
from had this support (lib/signal.c, W. Richard Stevens' `UNP 2nd ed Vol 1').
* remove <termios.h> from util.c; it should be unnecessary now
 1.36 26-Sep-1999  lukem * replace ifdefs against __SVR4 and __linux__ with DIRENT_MISSING_D_NAMLEN;
it's more portable and more obvious
* remove the mkgmtime() && HAVE_TIMEGM stuff:
a) why should netbsd have to define HAVE_TIMEGM to compile cleanly?
b) foreign compiles of ftp should just be linked with working
timegm function

a more portable version of this ftp client will be released as a 3rdparty
product; no use polluting our code with half-baked attempts...
 1.35 22-Sep-1999  lukem branches: 1.35.2;
* add support for `xferbuf', which sets both `sndbuf' and `rcvbuf'
* document the above three commands
* rototill the way the sndbuf and rcvbuf work. remove resetsockbufsize()
* use the appropriate socket buffer size as the size of the buffer that
the read()/write() loops use. speeds up things in some cases.
 1.34 12-Jul-1999  lukem * change fetch_ftp() to be fully rfc 1738 compliant; if the URL contains
the an empty directory (e.g, between `some' and `path' in
`ftp://host/some//path'), then execute `CWD ' (without a path).
This command will probably fail on rfc 959 compliant servers, so
issue a warning in this case and bail. [noted by cgd].
(i wonder if the people who wrote rfc 1738 actually realised that this
requirement appears to contravene the spec for `cwd' in rfc 959 ?)
* replace isurl() with isipv6addr(), and use appropriately. fixes
auto-login with `classic ftp URLs' (e.g, `ftp somehost:')
* cleanup and rework some of the ipv6 stuff in parse_url()
* prevent potential coredump in fetch_ftp() when parsing `;type=X'
* KNF a few lines
* fix a couple of comments
* cleanup the man page a bit
 1.33 11-Jul-1999  itojun add epsv4 command, which enable/disable the use of EPSV/EPRT.

this is mainly for (hypothetical) ftp server which disconnect clients
that use EPSV/EPRT. I've never seen any ftp server like this, but
epsv4 command may be of use when such an ftp server is found.
 1.32 02-Jul-1999  itojun add dual-stack (IPv4/v6) support. hope I broke no other part...
 1.31 29-Jun-1999  lukem [fear this; more ftp hacking from lukem :-]

features:
 1.30 24-Jun-1999  christos remove declaration for empty; it is only used in one file and hide
it's implementation.
 1.29 20-Jun-1999  cgd split the "SMALL" #ifdefs into ones for NO_ABOUT, NO_EDITCOMPLETE, and
NO_PROGRESS. -DSMALL still implies all of those. progress meter support
isn't necessary for the smallest possible ftp client, but it adds very
little space and makes users' lives much better. Therefore, it should
be enabled for installation media if at all possible.
 1.28 22-Mar-1999  lukem branches: 1.28.2;
* implement -R; restart non-proxied command-line FTP xfers
* fix fetch_ftp() so that hcode parsing is not done for file:// urls
(a } in the wrong place, and code at the wrong indent level...)
* change outfile to being a global (so it gets correctly reset)
* change parse_url to not remove leading '/' for non ftp urls.
whilst this is not totally rfc1738 compliant, other code kinda
assumes this is the case, and it doesn't hurt
 1.27 07-Feb-1999  lukem changes to fix [bin/6951] by Peter Simons <simons@cys.de>:
* implement ftpvis(), which \ quotes the following in the given string:
SP, TAB, \, CR, "
* use ftpvis() in complete_ambiguous(), to escape characters in a word
which would confuse makeargv()/slurpstring().
 1.26 18-Nov-1998  itohy Temporary fix of UTC -> time_t conversion:
1. changed to use timegm(3) on NetBSD and
2. supply alternative code for other Unix-like OSs
(NetBSD ftp shall be portable for some reasons :-).

This fix closes PR #6448.

NOTE: This should be fixed again if a portable UTC to time_t
conversion method is specified in some standards.
 1.25 08-Aug-1998  lukem * implement xsignal(); same semantics as signal() but uses sigaction
with an explicit SA_RESTART. (needed for portability)
* use xsignal() for SIGALRM and SIGINFO handlers
 1.24 03-Aug-1998  lukem features:
* support $no_proxy, which is a comma or space separated list of
host[:port] elements for which proxying is to be disabled.
(asked for by cgd in [bin/5027])
* if $FTPANONPASS is defined, use that as the anon ftp password
(instead of "`whoami`@")
* allow http URL's without a filename as long as an output file
is specified.

other stuff:
* implement parse_url(), which breaks up a URL into its bits, and use.
* simplify url_get() and auto_fetch() to use parse_url() and to not
modify the supplied URL or a copy of it.
* implement xmalloc() and xstrdup(); error-checked malloc()/strdup()
* add more consistency to messages, quoting strings in output as `%s'
 1.23 26-Jul-1998  mycroft const poisoning.
 1.22 22-Jul-1998  lukem * send 'Connection: close' in HTTP/1.1 headers, preventing the
remove server from using a persistent connection, which speeds
up such requests.
* support http 301 and 302 redirects
* rewrite guts of url_get() to use fparseln() et al instead of
read(s,&p,1)... enables each in the header to be parsed
as necessary
* rename login to ftp_login, to remove conflict with util.h::login
* cleanup verbose messages during http proxy requests
 1.21 10-Jul-1998  thorpej Add "sndbuf" and "rcvbuf" commands for setting the socket buffer sizes,
which in turn can allow the use of larger TCP windows. This is a work in
progress; there is not yet support for specifying global defaults or
user prefrences on a host/network basis.
 1.20 04-Jun-1998  lukem some fixes & enhancements from openbsd's ftp, with extra fixes by me:
* default to passive with active fallback. $FTPMODE modifies this behaviour.
-A forces active connection.
* support '-o outfile' for auto-fetched files. outfile can be a file,
`-' (for stdout), or '|command' (to output each file through command).
* support '-r waittime', which retries the connection after waittime seconds
if it fails.
* fix 'page file' when restart is non-zero.
* try all ip-addresses of a host in a http fetch (as the normal ftp fetch
does).

XXX: a ``broken pipe'' error sometimes occurs with -o '|command';
i haven't tracked this down yet.
 1.19 18-Jan-1998  lukem * ensure buffer for username is initialised, so ^D on username prompt
doesn't use garbage for the username. from "Soren S. Jorvang" <soren@t.dk>
in [bin/4559]
* use in_port_t for ports, and USHRT_MAX instead of 0xffff
(from millert@openbsd.org)
* use `NULL' instead of `(.... *)0' where appropriate.
 1.18 01-Nov-1997  lukem * in recvrequest(), ignore restart_point unless "RETR"ieving. fixes problems
where a remote completion or `mget' would confuse the client a `restart'
had been issued beforehand. now, `restart' is remembered until an operation
that can actually use it is invoked.
* in sendrequest(), don't reset restart_point upon entry. fixes `restart'
for `put' operations.
* if `restart' is invoked with no arguments, print current setting instead
of displaying a usage
* consistently use printf("%qd", (long long)restart_point) when displaying
restart_point
* use strto[lq]() instead of atol() when parsing `mark' and `restart' values
* remove unnecessary strlen()s when result of previous snprintf() will do
* replace a few malloc()/strcpy()s with strdup()s
* use SECSPERHOUR instead of '3600'
 1.17 18-Aug-1997  lukem branches: 1.17.2;
bugs fixed:
* don't interpret '-' or '|' when a local filename is determined from
the remote name (i.e, in mget, and in get with only one argument).
This is implemented using an extra argument to recvrequest().
Fixes a major security hole.
* clean up memory leak when using globulize()
* clean up a couple of comments
* fix wording in TNF copyright

features added:
* support for TIS fwtk gate-ftp servers:
* read defaults from $FTPSERVER && $FTPSERVERPORT
* start in gate-ftp mode if invoked as 'gate-ftp'
* toggle or set with 'gate [host [port]]'
 1.16 20-Jul-1997  lukem * use RCSID() && COPYRIGHT() macros
* cleanup for WARNS=1 (including some ugly '(void)&var' bits wrapped in
#ifdef __GNUC__ to shut up gcc warnings WRT setjmp/longjmp)
* use strtol() instead of atoi(), and more extensively check result of
conversion
* use u_int16_t instead of short or int for TCP port addresses
 1.15 14-Apr-1997  lukem More enhancements/bugfixes (when will it end?)
* differentiate between being connected, and being logged in
* cleanup some text messages
* support username & password ftp URLs (ftp://user:pass@host/) in non-proxy
situations; assume proxy supports it for proxy situations.
* cd to / before performing any autofetch transfers
* use strncasecmp in URL parsing. fix from <Todd.Miller@courtesan.com>
 1.14 05-Apr-1997  lukem * support $ftp_proxy for ftp:// transfers [bin/3245]
* add "more" & "less" as synonyms for "page"
* move editline setup code into controlediting(), and call appropriately.
only setup setup terminal if going into interactive mode. inspired
by Todd Miller <todd.miller@courtesan.com>
 1.13 14-Mar-1997  christos SMALLFTP->SMALL; remove SMALL from Makefile
 1.12 13-Mar-1997  lukem Features:
* support remglobbing of auto_fetch arguments
* new flag - '-e'; disable editing
* "page file" == "get file |${PAGER-less}"

Bugfixes/cleanup:
* consistently use a trailing '.' on messages
* code cleanup, including buffer overrun fixes, use puts
and putchar in places, etc (inspired by OpenBSD mods)
* disable progress bar when local-file is a pipe or '-'
* skip \r in http headers
* fix remote ftpd slash bug more elegantly (so it works with ////)
* abort_remote(): check if cout==NULL before using it. should fix [bin/3273]
* fixed up cosmetic problems when complete_remote() generated errors from the
remote server (such as "no files found", "login with user and pass", ...)
done by adding extra argument to remglob(), which is a pointer to an error
buffer to put messages in rather than printing to stdout.
 1.11 01-Feb-1997  lukem [Yet Another Huge Ftp Commit - hopefully the last for a while,
barring any more little things people want added ...]

New features:
* progressmeter is now asynchronous, so "stalled" transfers can be
detected. "- stalled -" is displayed instead of the ETA in this case.
When the xfer resumes, the time that the xfer was stalled for is
factored out of the ETA. It is debatable whether this is better than
not factoring it out, but I like it this way (I.e, if it stalls for 8
seconds and the ETA was 30 seconds, when it resumes the ETA will still
be 30 seconds).
* verbosity can be disabled on the command line (-V), so that in auto-fetch
mode the only lines displayed will be a description of the file, and
the progress bar (if possible)
* if the screen is resized (and detected via the SIGWINCH signal), the
progress bar will rescale automatically.

Bugs fixed:
* progress bar will not use the last character on the line, as this can
cause problems on some terminals
* screen dimensions (via ioctl(TIOCWINSZ)) should use stdout not stdin
* progressmeter() used some vars before initialising them
* ^D will quit now. [fixes bin/3162]
* use hstrerror() to generate error message for host name lookup failure.
* use getcwd instead of getwd (it should have been OK, but why tempt fate?)
* auto-fetch transfers will always return a positive exit value upon failure
or interruption, relative to the file's position in argv[].
* remote completion of / will work, without putting a leading "///".
This is actually a bug in ftpd(1), where "NLST /" prefixes all names
with "//", but fixing every ftpd(1) is not an option...
 1.10 19-Jan-1997  lukem New features:
* Command line editing via editline(3) library.
* Context sensitive command and file completion, including remote files.

Enhancements to auto-fetch feature:
* Support for http:// URLs using the http protocol, including proxy HTTP
support via $htty_proxy if it's defined.
* The connection is kept open between successive files on the same host.
(obviously, this does not count for http requests.)
* Return value of ftp is 0 on no error, or the offset in argv[] of the file
which failed (i.e., argv[x] failed, ftp returns x).
* If the path in an ftp URL or classic format line has a trailing '/',
cd to the path and enter interactive mode. Fixes [bin/3011], albiet
requiring the user to help ftp in determining the operation.

Other changes:
* '-P port' works for normal ftp, and is the default for all classic style
auto-fetch transfers and for ftp URLs that don't specify the port.
(previously it would just work for the first xfer.)
* Some code moved into separated files along logical divisions.
* Editing and completion can be compiled out with -DSMALLFTP.
 1.9 09-Jan-1997  tls RCS ID police
 1.8 29-Dec-1996  lukem * preserve modtime if size is 0 [bin/3040, Enami Tsugutomo]
* in autofetch mode, don't retry a file if the connection failed - skip it
and move to the next file. [bin/3051, Matt Green]
* only print error messages from SIZE or MDTM if the user used 'size' or
'modt' (respectively). prevents extraneous warnings during normal transfers
when connected to a site which doesn't support these (behaviour prior to
last commit), but still allows error feedback to specific user requests
for this info (which the last commit broke).
* 'account': only accept one optional argument
* if invoked as pftp, default to passive mode on (from FreeBSD)
* remove leading '0 ' in progress bar - looked ugly
* use warn instead of perror
* use strncpy when src isn't known to have safe length
* remglob(): use mkstemp() to prevent symlink games, and don't override
_PATH_TMP, use it as the prefix to the temp file
 1.7 06-Dec-1996  lukem functionality mods:
* implement 'progress bar/meter' (inspired by ncftp). use 'progress' to
toggle on. it will display current file size to 5 digits, automatically
determining suffix (up to 16384 P (petabytes) == 2^64).
* 'ls' now uses NLST (unadorned listing), a la older ftp clients. 'dir'
still does LIST (long listing). idea from John Nemeth <jnemeth@cue.bc.ca>

bug fixes:
* return first line of reply in reply_string[] from getreply(), instead
of last line. This fixes [bin/741] (parsing of SYST), and also means
that SIZE and MDTM messages will be parsed correctly if they're longer
than 1 line.
* parse URL-style auto-ftps that have no filename correctly
(e.g, ftp://host, ftp://host/, ftp://host/dir/). pointed out by
Jaromir Dolecek <dolecek@saruman.ics.muni.cz>
* pass the correct size array in 2nd arg of utimes() when setting the
modification time
 1.6 28-Nov-1996  lukem More features, some of which were inspired by changes that
friedman@gnu.ai.mit.edu (Noah Friedman) made to his modified ftp:
- implement "lpwd" - local pwd
- implement "preserve" - toggle preserving of file modification
times on retrieved files
- allow for explicit "on" or "off" arg to toggle commands
- "exit" synonym for "quit", "msend" synonym for "mput"
- in confirmation mode, allow 'a' (yes to rest of current command),
and 'p' (turn off prompt mode, as if 'prompt off' was done,
effective immediately)
- "modtime" returns time formatted as localtime, not GMT

Bug fixes:
- check for extraneous args on commands
- cleanup const usage, line formatting
- create 0 length temporary file in remglob() to prevent symlink games
(from OpenBSD)
- check length of filename of ~/.netrc (from OpenBSD)
 1.5 25-Nov-1996  lukem cleanups/bugfixes:
- don't echo 'ACCT' parameter when debugging (a la 'PASS')
- Fix checking of directory access for "/foo", the parent
directory is "/", not "" (from FreeBSD)
- remove trailing whitespace on lines
- add any missing NetBSD tags
- cleanups to man page, includinging sorting options description

feature additions:
- variable sized hash marks (from [bin/683], but done in the hash command
as an optional arg)
- more user-friendly transfer time printing (from FreeBSD, with mods)
- '-p' command line option to jump into PASV mode (closes [bin/2857],
but with an option rather than checking argv[0])
- SIGINFO support for printing xfer stats when sending/receiving requests
- '-P port' for changing the port to connect to (from thorpej@netbsd.org)
- '-a': bypass normal login, and try anonymous login (from OpenBSD
via thorpej)
- autofetch files via url (ftp://...) or "classic" (host:/file)
(from OpenBSD via thorpej)
- 'ftp' synonymous with 'open' (from FreeBSD)
 1.4 08-Sep-1995  tls Sync with 4.4lite2
 1.3 29-Aug-1994  mycroft branches: 1.3.2;
Add RCS ids.
 1.2 25-Aug-1994  cgd passive mode support, from David Carrel <carrel@cisco.com>, and cleaned
up for the new ftp sources by me.
 1.1 25-Aug-1994  cgd branches: 1.1.1; 1.1.2;
Initial revision
 1.1.2.1 25-Aug-1994  cgd import from 4.4-Lite
 1.1.1.1 05-Sep-1995  tls imported from 44lite2
 1.3.2.2 29-Aug-1994  mycroft Add RCS ids.
 1.3.2.1 29-Aug-1994  mycroft file extern.h was added on branch netbsd-1-0 on 1994-08-29 03:09:11 +0000
 1.17.2.3 23-Nov-1998  cgd pull up rev 1.26 from trunk (itohy)
 1.17.2.2 10-Nov-1998  cgd pull up rev(s) 1.19-1.25 from trunk. (feyrer)
 1.17.2.1 18-Nov-1997  mellon Pull rev 1.18 up from trunk (lukem)
 1.28.2.2 19-Oct-2000  he Pull up revision 1.60 (via patch, requested by he):
Format string cleanup.
 1.28.2.1 25-Jun-1999  cgd pull up rev 1.29 from trunk. (cgd)
 1.35.2.1 27-Dec-1999  wrstuden Pull up to last week's -current.
 1.54.2.1 23-Jun-2000  minoura Sync w/ netbsd-1-5-base.
 1.57.2.1 18-Oct-2000  tv Pullup usr.bin string format fixes [is].
See "cvs log" for explicit revision numbers per file, from sommerfeld.
 1.60.2.1 10-Feb-2003  jmc Pullup rev 1.61 (requested by grant in ticket #1115)
Separate progress reporting into a utility set of routines. Use
these to provide a standalone progress binary and integrate support
for this into sysinst.
 1.64.2.5 24-Jul-2005  tron Pull up revision 1.69 (requested by lukem in ticket #606):
Implement:
int getline(FILE *stream, char *buf, size_t buflen, const char **errormsg)
Read a line from the FILE stream into buf/buflen using fgets(), so up
to buflen-1 chars will be read and the result will be NUL terminated.
If the line has a trailing newline it will be removed.
If the line is too long, excess characters will be read until
newline/EOF/error.
Various -ve return values indicate different errors, and errormsg
will be changed to an error description if it's not NULL.
Convert to use getline() instead of fgets() whenever reading user input
to ensure that an overly long input line doesn't leave excess characters
for the next input operation to accidentally use as input.
Zero out the password & account after we've finished with it.
Consistently use getpass(3) (i.e, character echo suppressed) when
reading the account data. For some reason, historically the "login"
code suppressed echo for Account: yet the "user" command did not!
Display the hostname in the "getaddrinfo failed" warning.
Appease some -Wcast-qual warnings. Fixing all of these requires
significant code refactoring. (mmm, legacy code).
 1.64.2.4 24-Jul-2005  tron Pull up revision 1.68 (requested by lukem in ticket #606):
Some const cleanups.
 1.64.2.3 24-Jul-2005  tron Pull up revision 1.67 (requested by lukem in ticket #606):
Fix some cast issues highlighted by Scott Reynolds using gcc 4 on OSX.4
 1.64.2.2 09-May-2005  tron Pull up revision 1.66 (requested by lukem in ticket #266):
gratuitous whitespace cleanup (before someone else jumps the gun...)
 1.64.2.1 09-May-2005  tron Pull up revision 1.65 (requested by lukem in ticket #265):
Implement a timeout on the accept(2) in dataconn() and the
connect(2) in xconnect() by temporarily setting O_NONBLOCK
on the socket and using xpoll() to wait for the operation
to succeed.
The timeout used is the '-q quittime' argument (defaults to
60s for accept(2), and the system default for connect(2)).
Idea inspired by discussion with Chuck Cranor.
This may (indirectly) fix various problems with timeouts
in active mode through broken firewalls.
Implement xpoll() as a wrapper around poll(2), to make it
easier to replace on systems without a functional poll(2).
Unconditionally use xpoll() instead of conditionally using
select(2) or poll(2).
 1.72.10.1 18-May-2008  yamt sync with head.
 1.74.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.75.6.1 13-May-2009  jym Sync with HEAD.

Third (and last) commit. See http://mail-index.netbsd.org/source-changes/2009/05/13/msg221222.html
 1.75.4.1 20-Nov-2010  riz Pull up following revision(s) (requested by lukem in ticket #1370):
usr.bin/ftp/extern.h: revision 1.78
usr.bin/ftp/fetch.c: revision 1.192
usr.bin/ftp/util.c: revision 1.153
Parse HTTP 'Date' entries in the `C' locale rather than the user's.
Fix from [bin/42917] (with minor changes), from KAMADA Ken'ichi.
 1.79.4.1 17-Dec-2013  bouyer Apply patch, requested by tron in ticket #997:
usr.bin/ftp/Makefile patch
usr.bin/ftp/cmds.c patch
usr.bin/ftp/cmdtab.c patch
usr.bin/ftp/extern.h patch
usr.bin/ftp/fetch.c patch
usr.bin/ftp/ftp.1 patch
usr.bin/ftp/ftp.c patch
usr.bin/ftp/ftp_var.h patch
usr.bin/ftp/main.c patch
usr.bin/ftp/progressbar.c patch
usr.bin/ftp/ssl.c patch
usr.bin/ftp/ssl.h patch
usr.bin/ftp/util.c patch
usr.bin/ftp/version.h patch

Add HTTPS support to ftp(1).
 1.79.2.1 30-Oct-2012  yamt sync with head
 1.80.32.2 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.80.32.1 10-Jun-2019  christos Sync with HEAD
 1.80.24.3 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1763:

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/cmds.c up to 1.141
usr.bin/ftp/complete.c up to 1.47
usr.bin/ftp/domacro.c up to 1.23
usr.bin/ftp/extern.h up to 1.82
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp.c up to 1.174
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/progressbar.c up to 1.24
usr.bin/ftp/progressbar.h up to 1.9
usr.bin/ftp/ssl.c up to 1.12
usr.bin/ftp/ssl.h up to 1.5
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

ftp(1): validate address from PASV and LPSV response.
ftp(1): use raw write(2) instead of fwrite(3) to avoid stream
corruption because of the progress bar interrupts.
Fixes for PR 56219 and PR 55857.
PR 57003: Support relative redirects.
 1.80.24.2 12-Sep-2022  martin Backout ticket #1763 for now - trust anchors are not solved.
 1.80.24.1 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1763:

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/cmds.c up to 1.141
usr.bin/ftp/complete.c up to 1.47
usr.bin/ftp/domacro.c up to 1.23
usr.bin/ftp/extern.h up to 1.82
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp.c up to 1.174
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/progressbar.c up to 1.24
usr.bin/ftp/progressbar.h up to 1.9
usr.bin/ftp/ssl.c up to 1.11
usr.bin/ftp/ssl.h up to 1.5
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

ftp(1): validate address from PASV and LPSV response.
ftp(1): use raw write(2) instead of fwrite(3) to avoid stream
corruption because of the progress bar interrupts.
Fixes for PR 56219 and PR 55857.
PR 57003: Support relative redirects.
 1.82.10.2 02-Dec-2024  martin Pull up following revision(s) (requested by lukem in ticket #1021):

usr.bin/ftp/ftp.c: revision 1.178
usr.bin/ftp/version.h: revision 1.99
usr.bin/ftp/ruserpass.c: revision 1.35
usr.bin/ftp/main.c: revision 1.134
usr.bin/ftp/main.c: revision 1.135
usr.bin/ftp/progressbar.c: revision 1.27
usr.bin/ftp/util.c: revision 1.165
usr.bin/ftp/cmds.c: revision 1.144
usr.bin/ftp/extern.h: revision 1.84
usr.bin/ftp/fetch.c: revision 1.242
usr.bin/ftp/ftp.1: revision 1.160

s/bninary/binary/ in comment.

extract duplicate code into a function.

Check bounds when copying to destination.

Remove const where the const string ended up being overwritten.

use unsigned when doing shifts.

remove const when string gets overwritten.

ftp: exit non-zero if short http transfer when filesize is known
If a http file size is known and the fetch finishes with less bytes
transferred, exit non-zero.
Bump version to 20241129.
PR bin/54713
PR bin/58281

ftp: help improvements
Document -? as a separate mode.
Document -H HEADER in the usage.
Clarify units for -b and -x.
Consistent argument names between ftp -? and ftp(1).

ftp: order getopt Upper before lower
Consistently order options in getopt and the switch
with the upper case option before the lower case option.
This makes it easier to cross-reference with -? and ftp(1).
No functional change.
 1.82.10.1 16-May-2023  martin Pull up following revision(s) (requested by lukem in ticket #170):

usr.bin/ftp/ssl.c: revision 1.13
usr.bin/ftp/fetch.c: revision 1.236
usr.bin/ftp/util.c: revision 1.166
usr.bin/ftp/main.c: revision 1.129
usr.bin/ftp/extern.h: revision 1.83
usr.bin/ftp/ftp.1: revision 1.148
usr.bin/ftp/cmdtab.c: revision 1.53
usr.bin/ftp/version.h: revision 1.96

Add option sslnoverify to control validation of SSL certificates.

Add netrc processing to fetch-mode (URL on command line) to enable options and autologin
via netrc.

Fix SSL cleanup in some error paths.

Certificate validation is now enabled by default. Set FTPSSLNOVERIFY=1 in environment
or configure a corresponding init macro via netrc to not validate certs (required if
you haven't installed a required CA certificate for OpenSSL).

Discussed with lukem@ on icb.
 1.83.2.1 02-Aug-2025  perseant Sync with HEAD
 1.242 29-Nov-2024  lukem ftp: exit non-zero if short http transfer when filesize is known

If a http file size is known and the fetch finishes with less bytes
transferred, exit non-zero.

Bump version to 20241129.

PR bin/54713
PR bin/58281
 1.241 25-Sep-2024  christos PR/58581: Sunil Nimmagadda: Add flag to allow specifying extra http header
fields.
 1.240 25-Sep-2024  christos pass some lint.
 1.239 18-Feb-2024  christos branches: 1.239.2;
Add -b <buflen> to specify the buffer size.
 1.238 12-Aug-2023  mlelstv Don't finish downloading an empty file with 'already done' before it is
created locally.
 1.237 02-Jul-2023  mlelstv Fix HTTPS through Proxy.

While a regular HTTP Proxy, requires the absolute URL with protocol
and host part, yyou must only send the relative URL through a
CONNECT tunnel (you are talking to the target server).
 1.236 25-Feb-2023  mlelstv Add option sslnoverify to control validation of SSL certificates.
Add netrc processing to fetch-mode (URL on command line) to enable options and autologin
via netrc.
Fix SSL cleanup in some error paths.

Certificate validation is now enabled by default. Set FTPSSLNOVERIFY=1 in environment
or configure a corresponding init macro via netrc to not validate certs (required if
you haven't installed a required CA certificate for OpenSSL).

Discussed with lukem@ on icb.
 1.235 11-Sep-2022  christos branches: 1.235.2;
PR/57003: Handle relative URLs (patch by kim@)
 1.234 01-Aug-2021  andvar fix typos in word "otherwise".
 1.233 06-Jul-2021  christos Use raw write(2) instead of fwrite(3) to avoid stream corruption because
of the progress bar interrupts. From RVP.
 1.232 11-Jul-2020  lukem fetch_url: improve signal handler restoration

Use SIG_ERR not NULL as the indicator that a signal handler
hasn't been changed, so that SIG_DFL (equivalent to NULL)
will be restored.

Fix restoration of SIGQUIT; use the old handler not SIGPIPE's.
 1.231 04-Apr-2019  christos branches: 1.231.2;
Make fetch_read() return size_t like fread() does. It is bogus to
have one backing implementation that returns different values and
types than the other. Handle error setting properly; i.e. bail
out if the internal read returned an error. Now we get a proper
error message when the the server resets our connection instead of
a warning that the right failed with an invalid argument.

The server used for testing was:
http://capeweather.dyndns.org:8080/graphs/3474.png
Which seems to be unreliable :-)
 1.230 11-Feb-2018  christos branches: 1.230.4;
more volatile to appease gcc.
 1.229 25-Nov-2017  christos Make outfile always allocated, free it to set it to NULL, and don't move it
around.
 1.228 15-Feb-2017  nonaka branches: 1.228.4;
ftp(1): split the auth processing function.
 1.227 31-Jan-2017  christos Use the first name we requested the http/https URL for, not any name we ended
up with after random redirects.
 1.226 15-Dec-2016  nonaka branches: 1.226.2;
handle proxy authentication correctly.
 1.225 17-Oct-2016  christos PR/51558: ast@: ftp dumps core after usage message when IPv6 URL lacks a slash.
Initialize variable so that we don't get random behavior on cleanup.
 1.224 03-Aug-2016  maya Do globbing for FTP URLs of the form ftp://... too

ok christos
 1.223 04-Apr-2016  christos branches: 1.223.2;
PR/51043: Yorick Hardy: ftp(1) should use the port number for CONNECT
 1.222 18-Mar-2016  christos sprinkle more volatile (distribution build with gcc-5.3)
 1.221 05-Feb-2016  nonaka Initialize the token match pointer.
 1.220 05-Jan-2016  wiz Fix downloads of local files using file:// URLs

Previously it would error out in copyurlinfo() when copying a NULL port.
 1.219 17-Dec-2015  christos mark function as only needed with ssl.
 1.218 17-Dec-2015  christos Split the position/size parsing into a separate function.
 1.217 17-Dec-2015  christos Simplify and factor out connect message
 1.216 17-Dec-2015  nonaka - Fix to connect https via proxy.
- Fix ttyout message.
 1.215 16-Dec-2015  christos PR/50438: NONAKA Kimihiro: ftp(1): CONNECT method support
Please test!
 1.214 16-Dec-2015  christos more refactoring:
- introduce authinfo and urlinfo structures
- split negotiation code out.
 1.213 16-Dec-2015  nonaka Fix compile failure without WITH_SSL.

> /tmp/bracket/build/2015.12.15.21.01.27-i386/src/usr.bin/ftp/fetch.c: In function 'fetch_url':
> /tmp/bracket/build/2015.12.15.21.01.27-i386/src/usr.bin/ftp/fetch.c:823:18: error: 'HTTPS_URL_T' undeclared (first use in this function)
> urltype == HTTPS_URL_T ? &ssl : NULL);
 1.212 15-Dec-2015  christos Factor the proxy handling code out.
 1.211 15-Dec-2015  christos Separate no_proxy handling.
 1.210 15-Dec-2015  christos Try to factor out some code, this is completely out of control.
 1.209 13-Dec-2015  tron (Hopefully) fix build without IPv6 support
 1.208 11-Dec-2015  tron Use the proper format "[IPv6 address]:port" when reporting connection
attempts to IPv6 endpoints.
 1.207 12-Sep-2015  wiz Add Server Name Indication (SNI) support for https.

Needed for e.g. some github URLs.
 1.206 26-Oct-2014  christos don't pay attention to special characters if they don't come from the command
line (from jmcneill)
 1.205 07-Nov-2013  christos branches: 1.205.4;
more volatile for m68k
 1.204 03-Nov-2013  christos let progressmeter deal with the timeout once we've started transferring.
 1.203 02-Nov-2013  christos PR/34796: Hauke Fath: ftp does not timeout on http fetches.
 1.202 23-Feb-2013  christos fix restart from anon ymous
 1.201 22-Dec-2012  christos compile without SSL
 1.200 22-Dec-2012  christos document https support, mentioning certificates are not validated, and obey
https_proxy.
 1.199 21-Dec-2012  christos PR/47276: Add https support
 1.198 04-Jul-2012  is branches: 1.198.2;
As discussed on tech-net@: Don't display expected EHOSTUNREACH for all but
the last connect attempts in terse mode.
 1.197 24-Feb-2012  apb subtracting two pointers yields ptrdiff_t, so cast it to int.
 1.196 24-Feb-2012  apb When given an URL that contains "://" but is not recognised,
print an error message. Now "ftp https://foo/bar" prints

ftp: Unsupported URL scheme `https'

instead of

ftp: Can't lookup `https:ftp': No address associated with hostname
ftp: Can't connect or login to host `https:?'
 1.195 10-Dec-2011  lukem branches: 1.195.2; 1.195.6; 1.195.8;
Move determination of socket buffer sizes from startup to the first
time a socket is used, as the previous logic assumed AF_INET sockets
were available (which they may not be in an IPv6-only system).
Per discussion with Maxim Konovalov and the FreeBSD problem 162661.
 1.194 16-Sep-2011  joerg branches: 1.194.2;
Use __dead
 1.193 05-Mar-2010  lukem Back to using 'RFC xxxx' instead of 'RFCxxxx'
 1.192 04-Mar-2010  lukem Parse HTTP 'Date' entries in the `C' locale rather than the user's.
Fix from [bin/42917] (with minor changes), from KAMADA Ken'ichi.
 1.191 17-Aug-2009  christos back out previous; luke says:
'@' is a reserved URI char per RFC 3986, use %40
 1.190 16-Aug-2009  christos use strrchr to find the last @ because we might want the username to contain
user@domain.
 1.189 13-Aug-2009  drochner avoid NULL dereference in log output if the command line parser
failed to extract a port number from the URL
 1.188 13-Jul-2009  roy Rename internal getline() function to get_line() so it does
conflict with the soon to be added getline(3) libc function.
 1.187 12-Apr-2009  lukem Fix numerous WARNS=4 issues (-Wcast-qual -Wsign-compare).
 1.186 12-Apr-2009  lukem fix -Wshadow issues
 1.185 28-Apr-2008  martin branches: 1.185.6; 1.185.8; 1.185.14;
Remove clause 3 and 4 from TNF licenses
 1.184 22-Apr-2008  lukem Use the service name to getaddrinfo() (along with the host name), so that
features such as DNS Service Discovery have a better chance of working.
Suggested by David Young <dyoung>.

Display the service name in various status & error messages.

Don't getservbyname() the :port component of a URL; RFC 3986 says it's
just an unsigned number, not a service name.
 1.183 05-Dec-2007  lukem branches: 1.183.6;
DPRINTF() consistency tweaks
 1.182 22-Aug-2007  lukem branches: 1.182.2;
Cast the field precision calculation to int.
Should fix the amd64 build problem noticed by Paul Goyette.
 1.181 22-Aug-2007  lukem Improve parsing of chunked transfer chunks per RFC2616:
* more stringent chunk-size parsing
* ignore optional trailing ';chunk-ext' stuff, instead of barfing
* detect EOF before final \r\n.
 1.180 05-Jun-2007  lukem Enforce restriction that (http) proxied URL fetchs don't support
being restarted at this time.
PR #28697.
 1.179 24-May-2007  lukem Display times in RFC2822 form rather than using ctime(3), since
the former is more explicit about the timezone offset.
 1.178 22-May-2007  lukem * main: call tzset() to ensure TZ is setup for other <time.h> functions.
* remotemodtime(): use strptime() to parse the reply.
* fetch_url(): ensure struct tm is zeroed before calling strptime().
 1.177 15-May-2007  lukem * Modify parse_url() to consistently strip the leading `/' off ftp URLs.
Fixes PR 17617.
* Use 'RFCnnnn' (with leading 0) instead of 'RFC nnnn', to be
consistent with the style in the RFC index.
* Refer to RFC3916 instead of 1738 or 2732.
* Expand the list of supported RFCs in ftp(1) to contain the document
name as well.
 1.176 10-May-2007  lukem Switch from fparseln() to the internal getline() when parsing HTTP headers.
Makes ftp a bit more portable (not needing fparseln()) at the expense of not
supporting arbitrary long header lines, which I'm not concerned about
because we don't support header line continuation either...
 1.175 17-Apr-2007  lukem * Implement -s srcaddr; uses srcaddr as the local IP address for all
connections.
Based on code in the version of ftp that FreeBSD had before they
replaced it with lukemftp.
* Move error message handling into ftp_connect() rather than in the
caller, so that more specific error reporting can occur.
* Improve consistency of various warning and error messages.
 1.174 11-Apr-2007  lukem getpass() can return NULL upon error in some implementations
(as documented in older standards documents, before the API was obsoleted).
Problem observed in tnftp on Solaris by Emil Mikulic.
 1.173 13-Dec-2006  christos gcc4 does not care about &foo; use volatile instead. From Anon Ymous
 1.172 25-Nov-2006  christos spell precede; from Zafer
 1.171 22-Sep-2006  elad PR/34540: b1ff at fr33 dot b33r dot net: bogus free() in ftp(1)
Applied patch, thanks!
 1.170 26-Jul-2006  lukem If a file upload (via -u) fails, return an non-zero exit value based on the
index of the file that caused the problem (a la auto-fetch retrieval).
Problem noted by A P Garcia in private email.
 1.169 28-Apr-2006  christos Coverity CID 2194: Don't forget to free port.
 1.168 28-Apr-2006  christos Coverity CID 2195: Free path in all code paths, not some.
 1.167 31-Jan-2006  christos rename debug to ftp_debug. grr libssh.
 1.166 31-Jan-2006  christos Rename xfoo() to ftp_foo() to avoid collisions with libssh. Don't ask.
 1.165 02-Jan-2006  christos recognize 307 redirect code.
 1.164 21-Aug-2005  lukem Don't clear the trailing character on the auth_url() username;
we now use getline() and that newline strips for us.
Problem found & fixed by Mark Davies.
 1.163 29-Jun-2005  christos Add NO_USAGE and NO_DEBUG so that we can fit in the floppies again.
 1.162 10-Jun-2005  lukem Implement:
int getline(FILE *stream, char *buf, size_t buflen, const char **errormsg)
Read a line from the FILE stream into buf/buflen using fgets(), so up
to buflen-1 chars will be read and the result will be NUL terminated.
If the line has a trailing newline it will be removed.
If the line is too long, excess characters will be read until
newline/EOF/error.
Various -ve return values indicate different errors, and errormsg
will be changed to an error description if it's not NULL.

Convert to use getline() instead of fgets() whenever reading user input
to ensure that an overly long input line doesn't leave excess characters
for the next input operation to accidentally use as input.

Zero out the password & account after we've finished with it.

Consistently use getpass(3) (i.e, character echo suppressed) when
reading the account data. For some reason, historically the "login"
code suppressed echo for Account: yet the "user" command did not!

Display the hostname in the "getaddrinfo failed" warning.

Appease some -Wcast-qual warnings. Fixing all of these requires
significant code refactoring. (mmm, legacy code).
 1.161 01-Jun-2005  lukem * Only print the "Trying <address>..." message if verbose and
there's more than one struct addrinfo in the getaddrinfo() result.
* Don't use non-standard "u_int".
 1.160 29-May-2005  lukem * fetch_ftp(): preserve 'anonftp' across a disconnect() so that multiple
ftp auto-fetches on the same command line login automatically.
* auto_fetch(): use an initialized volatile int to appease IRIX cc.
 1.159 19-May-2005  lukem Use size_t instead of int where appropriate.
 1.158 14-May-2005  lukem Fix some cast issues highlighted by Scott Reynolds using gcc 4 on OSX.4
 1.157 11-Apr-2005  lukem gratuitous whitespace cleanup (before someone else jumps the gun...)
 1.156 10-Apr-2005  lukem In fetch_url(), don't call freeaddrinfo(res0) too early, as we use pointers
to its contents later in the function.
Problem found by Onno van der Linden.
 1.155 12-Jan-2005  lukem branches: 1.155.2;
Fix ftp url reget when globs are being used.
Provided by Mathieu Arnold <mat@FreeBSD.org>.
 1.154 10-Dec-2004  lukem * Always decode %xx in a url's user & pass components.
* Only remember {WWW,Proxy}-Authenticate "Basic" challenges; no point
in tracking any others since ftp doesn't support them.
* Improve the parsing of HTTP responses.
 1.153 30-Oct-2004  dsl Add (unsigned char) cast to ctype functions
 1.152 08-Aug-2004  lukem Don't base64 encode the trailing NUL in the HTTP basic auth response.
Problem noted by Eric Haszlakiewicz.
 1.151 21-Jul-2004  lukem Slightly rework SIGINT handling; if we're exiting the auto-fetch stuff
and sigint_raised is non-zero, reset the handler for SIGINT to SIG_DFL
and raise(SIGINT) so that the appropriate wait(3) status is setup.
Based on solution proposed by Ognyan Kulev.
This should really fix PR [pkg/26351].
 1.150 20-Jul-2004  lukem Improve parsing of HTTP response headers to be more RFC2616 compliant, and
skip LWS (linear white space; CR, LF, space, tab) and the end of lines and
between the field name and the field value. This still isn't 100% compliant,
since we don't support "multi line" responses at this time.
This should fix PR [bin/22611] from TAMURA Kent (although I can't easily
find a http server to reproduce the problem against.)

Fix a minor memory leak when parsing HTTP response headers.
 1.149 20-Jul-2004  lukem Don't unnecessarily display a 401/407 error when running with -V.
Fix from PR [bin/18535] by Jeremy Reed.
 1.148 20-Jul-2004  lukem If an ftp auto-fetch transfer is interrupted by SIGINT (usually ^C),
exit with 130 instead of 1 (or rarely, 0).
This allows an ftp auto-fetch in a shell loop to correctly terminate the loop.
Should fix PR [pkg/26351], and possibly others.
 1.147 06-Jun-2004  christos Save approximately 8K by not including http authentication, extended status
messages and help strings when the appropriate options are set.
 1.146 10-Dec-2003  lukem Don't warn about "ignored setsockopt" failures unless debugging is
enabled. Suggested by Todd Vierling.

Allow empty passwords in ftp://user:@host/file auto-fetch URLs,
per RFC 1738. Requested by Simon Poole.

Update version.
 1.145 04-Dec-2003  lukem correct URL syntax in comment
 1.144 31-Jul-2003  lukem * $FTPUSERAGENT overrides the HTTP User-Agent header.
Based on patch from Douwe Kiela.
* Add about:tnftp
* Fix URL in about:netbsd
* Crank version
 1.143 26-Jul-2003  salo netbsd.org->NetBSD.org
 1.142 12-Jul-2003  itojun strlcpy
 1.141 14-May-2003  wiz Uppercase URL.
 1.140 07-Mar-2003  grant fix a bogus error message when given a HTTP URL with a trailing
slash.
 1.139 28-Feb-2003  lukem crank dates
 1.138 28-Feb-2003  lukem add a cast to appease SUNWspro cc. noted by grant@
 1.137 30-Nov-2002  lukem tweaks for fparseln(3) move from libutil to libc:
- remove #include <util.h> if nothing else needed it
- remove LDFLAGS+=-lutil if nothing else needed it
 1.136 05-Jun-2002  lukem - when showing the final progress bar, replace "00:00 ETA" with the
elapsed time. (suggested by simonb)
- actually display transfer stats after a URL fetch. (bug introduced a
*long* time ago)
- update copyright & version
 1.135 06-May-2002  lukem Handle URLs without files correctly (e.g, when using '-o -').
Fix from Anders Dinsen <anders@dinsen.net> in [bin/13768]
 1.134 06-May-2002  lukem - Only send port number in HTTP/1.1 Host: request if port != 80.
Fixes [bin/15415] from Takahiro Kambe <taca@sky.yamashina.kyoto.jp>
- Fix bad mode passed by mls() to recvrequest().
Fixes [bin/16642] from <steve.mcclellan@radisys.com>
 1.133 01-Feb-2002  itojun portnum is unsigned, use %u instead of %d
 1.132 26-Dec-2001  lukem update copyrights
 1.131 23-Dec-2001  lukem Add -4 to force IPv4 and -6 to force IPv6 address usage.
From Hajimu UMEMOTO, via Mike Heffner of FreeBSD.

(FreeBSD has imported NetBSD's ftp as their ftp client;
Mike is sending back some of their local changes).
 1.130 29-Nov-2001  lukem use u_char instead of char in base64_encode().
problem noticed by Jorgen Lundman in private mail.
 1.129 25-Nov-2001  yamt don't make broken file with -R option.
 1.128 25-Nov-2001  yamt handle "*" in Content-Range properly.
 1.127 15-Oct-2001  tacha If no_proxy condition is true && urltype == FTP_URL_T, use fetch_ftp to retrieve.
 1.126 19-Feb-2001  cgd convert to use getprogname()
 1.125 28-Sep-2000  lukem explicitly use SOCK_STREAM with socket() instead of res->ai_socktype,
because it appears that linux with glibc doesn't set the latter
correctly after one of getaddrinfo() or getnameinfo().
 1.124 28-Aug-2000  lukem base64_encode should be static. picked up by hp/ux(!) compiler
 1.123 27-Aug-2000  lukem It appears that whilst Apache 1.3.9 incorrectly puts a trailing space
after the chunksize (before the \r\n), Apache 1.3.11 puts *multiple*
trailing spaces after the chunksize. I 'm fairly certain that this is
contrary to RFC 2068 section 3.6, but whatever...
Found by David Brownlee <abs@mono.org>
 1.122 06-Aug-2000  lukem * implement parseport(), which takes a string and attempts to convert
it to a numeric port number
* use parseport() in parse_url() and hookup()
* don't try and lookup the port number using getaddrinfo(), as it's too hard
to separate a failed host name lookup from a failed service name lookup.
this was causing lossage on systems that don't have `http' in services(5)
(such as solaris), but only crept in when we started using getaddrinfo()
unconditionally.
 1.121 01-Aug-2000  lukem - rename NO_QUAD to NO_LONG_LONG, QUAD* -> LL* and add ULL* (unsigned)
equivalents. name change suggested by Klaus Klein <kjk@netbsd.org>
- change defined(BSD4_4) || HAVE_SIN_LEN tests into HAVE_SOCKADDR_SA_LEN,
and set the latter if BSD4_4 exists
 1.120 30-Jul-2000  lukem * always use getaddrinfo() and getnameinfo() instead of maintaining two code
paths. (lukemftp will provide replacements for these on older systems)
* rename __USE_SELECT to USE_SELECT
* rename BSD4_4 to HAVE_SIN_LEN
* replace union sockunion {} with struct sockinet {}, and modify the code
accordingly. this is possibly more portable, as it doesn't rely upon the
structure alignment within the union for our own stuff.
(XXX: haven't tested the ipv6 stuff)
 1.119 30-Jul-2000  lukem clean up NO_QUAD support: create helper #defines and use as appropriate:
#define NOQUAD ! NOQUAD
------- ------ - ------
QUADF "%ld" "%lld"
QUADFP(x) "%" x "ld" "%" x "lld"
QUADT long long long
STRTOL(x,y,z) strtol(x,y,z) strtoll(x,y,z)
 1.118 18-Jul-2000  lukem minor knf. call setbinary()/setascii() with non-NULL 2nd arg
 1.117 07-Jul-2000  itojun errx?/warnx? audit. do not pass variable alone, use %s. idea from openbsd
 1.116 15-Jun-2000  lukem branches: 1.116.2;
* migrate the SYST parsing from setpeer() into a separate remotesyst().
call remotesyst() only when login has been successful
some servers don't let you run SYST until you've successfully logged in.
* in fetch_ftp(), always call setpeer() with autologin disabled, and use
the following ftp_login() to DTRT.
this prevents ftp from trying to login a second time if the first autologin
fails when connecting to a remote site anonymously using autofetch.
* reset unix_proxy and unix_server in cleanuppeer()
* missed a function conversion in the KNF sweep...
 1.115 05-Jun-2000  lukem - fix ai_unmapped() to be a no-op in the !def INET6 case
- display `(-INET6)' at the end of the version string if !def INET6
- clarify in the man page that IPv6 support may not be present (for lukemftp :)
 1.114 31-May-2000  lukem Add support for 'ftp -u url file ...', to upload a list of files to given url.
Mostly based on [bin/10019] by Scott Aaron Bamford <sab@ansic.net>
 1.113 29-May-2000  itojun convert IPv4 mapped address (::ffff:10.1.1.1) into real IPv4 address
before touching it. IPv4 mapped address complicates too many things
in FTP protocol handling.
 1.112 25-May-2000  itojun branches: 1.112.2;
do not pass scoped IPv6 address notation on Host: directive, since
scope identifier is local to the originating node.
do not allow scoped IPv6 address notation in URL, if it is via proxy.
 1.111 01-May-2000  lukem convert to ANSI KNF
 1.110 01-May-2000  lukem * Add support for `fget localfile', which reads a list of filenames to
retrieve from localfile. Based on work by Darren Reed.
* Crank version.
* Update copyright dates.
 1.109 13-Apr-2000  lukem fixes from cgd:
* sanity check a length (otherwise certain bogus responses can crash ftp)
* allow a transfer encoding type of `binary'; certain firewall vendors
return this bogus type...
 1.108 13-Apr-2000  lukem s/strtoq/strtoll/ (the latter is standardised)
 1.107 09-Mar-2000  itojun make debugging output unambiguous on IPv6 numeric addrs (don't use host:port)
 1.106 09-Mar-2000  itojun http://[::1]:8080/ is legal.

send Host: directive with RFC2732 bracket notation for IPv6 numeric,
otherwise "host:port" is ambiguous to servers (clarification will be submitted
as update to RFC2732).
 1.105 14-Feb-2000  lukem only use getaddrinfo() et al if both NI_NUMERICHOST *and* INET6 are defined...
(allows --disable-ipv6 in lukemftp's configure script to disable this as
well, which is good for testing when it appears getaddrinfo() is borken)
 1.104 31-Jan-2000  lukem define private type `sigfunc' as
typedef void (*sigfunc) __P((int));
and replace use of sig_t and void (*)(int).

certain other OSes define sig_t differently to that (they add extra arguments),
and it causes problems due to function mismatches, etc...
 1.103 25-Jan-2000  lukem work around bug in apache 1.3.9 which incorrectly puts a trailing
space after the chunksize.
noted by Jun-ichiro itojun Hagino <itojun@itojun.org> in [bin/9096]
 1.102 21-Dec-1999  lukem change references from draft-ietf-ipngwg-url-literal-01.txt to RFC 2732
 1.101 11-Dec-1999  lukem Fix chunked support; probably broke after rate limiting was added.
Problem noticed/debugging assisted by giles lean <giles@nemeton.com.au>.

XXX: rate limiting with chunked xfers might not limit correctly (i.e,
the limit may be too high or too low); fixing this is non trivial,
and will probably occur if i ever rototill fetch_url()
 1.100 05-Dec-1999  lukem move version into separate file to reduce recompilation after version crank.
 1.99 05-Dec-1999  lukem * remove unnecessary freeaddrinfo(res), since res0 was changed to be
freed earlier in itojun's last commit. fixes [bin/8948].
* remove `const char *reason'; it was being assigned but not used.
 1.98 03-Dec-1999  itojun fix memory leak in fetch_url (no freeaddrinfo was there).
sync with recent KAME.
 1.97 26-Nov-1999  lukem * complete_remote(): use remglob("", ...) instead of remglob(".", ...),
for listings of the current working directory; some ftp servers don't
like `NLST .'.
[noted by Giles Lean <giles@nemeton.com.au>]
* recvrequest(): treat remote=="" as remote==NULL when calling command().
(to support the above change)
* support `[user@]' in `[user@]host' and `[user@]host[:][path]'.
[based on idea (and initial code) from David Maxwell <david@fundy.ca>]
* `idle' may be invoked without any args
* reformat some comments
* reformat usage string in program and man page
* call updateremotepwd() after successful login, not after successful connect
* always call setsockopt(, IPPROTO_IP, IP_TOS, ) (et al); using #if
defined(IPPROTO_IP) doesn't work on certain foreign systems where
enums instead of #defines are used...
[noted by Matthias Pfaller <leo@dachau.marco.de>]
 1.96 11-Nov-1999  lukem whitespace nits
 1.95 10-Nov-1999  lukem - implement restarting file:/// non-proxied http:// URLs (with -R).
- fix a semicolono which stopped file:/// from working
 1.94 09-Nov-1999  lukem - split the version string into product and version
- be consistent about reporting the version between:
+ status command
+ about:version URL fetch
+ User-agent sent in http requests
 1.93 09-Nov-1999  lukem when using http/1.1 for unproxied http requests, send the port as a number
(e.g, `:80') instead of a name (e.g, `:http'), because the former is the
supported method that actually works... noted by hubertf@netbsd.org
 1.92 03-Nov-1999  lukem support `about:version'. also display the version in the output of `status'.
 1.91 24-Oct-1999  lukem new features:
- add `usage'; displays the usage of a command.
implemented by calling the c_handler() with argc = 0, argv = "funcname".
- add `passive auto'; does the same as $FTPMODE=auto.
- add `set [option value]'; display all options, or set an option to a value.
- add `unset option'; unset an option.
- add getoptionvalue() to retrieve an option's value, and replace a few
global variables with calls to this.
- implement cleanuppeer(), which resets various bits of state back to
`disconnected'. call in disconnect() and lostpeer().
- support completing on `options'.
- improve recovery after a SIGINT may have closed the connection.
XXX: there's still a couple to fix

other stuff:
- various consistency fixes in the man page.
- ensure that the command usage strings in the code and man page match reality.
- mput/mget: check that the connection still exists before each xfer.
- minor cosmetic changes in confirm().
- set code correctly in sizecmd() and modtime()
- don't need \n in err() strings.
- change lostpeer to take an argument (rather than casting (sig_t)lostpeer
in signal handlers)
- knf and whitespace police.
 1.90 12-Oct-1999  lukem a few user interface and cosmetic tweaks:
* confirm(): move from util.c to cmds.c. display mnemonic string in its prompt.
add support for `q' (terminate current xfer), `?' (show help list)
* in various signal handlers, output a linefeed only if fromatty.
* if fgets(stdin) returned NULL (i.e, EOF), clearerr(stdin) because you don't
want future fgets to fail. this is not done for the fgets() in the main
command loop, since ftp will quit at that point.
* unless ftp is invoked with -a, don't retain the anonftp setting between
hosts (`ftp somehost:' sets anonftp, but you don't want that to `stick'
if you close that connection and open a new one).
 1.89 10-Oct-1999  lukem use sigjmp_buf for sigsetjmp(), instead of jmp_buf.
noted by Havard.Eidnes@runit.sintef.no.
 1.88 09-Oct-1999  lukem * use sigsetjmp()/siglongjump() instead of setjmp()/longjmp(); the latter
don't save the signal mask on some foreign systems.
* ensure signal handlers don't use stdio and do reset errno if they
don't exit with siglongjmp()
* use a common SIGINT handler for {send,recv}request()
 1.87 06-Oct-1999  lukem more propaganda :)
 1.86 05-Oct-1999  lukem * fetch_url(): specifically set SIGQUIT to psummary before each xfer.
(work around editline's override)
* minor cleanup of signal handler (along the lines of similar work in
recvrequest()). the handlers should now be reset everytime the cleanup
handler was callled.
 1.85 05-Oct-1999  lukem * factor out SIGINFO setting into a handler that is always active (but only
prints out info if bytes > 0). only set the handler if SIGINFO is defined
* hijack SIGQUIT to be the same as SIGINFO (foreign ports have this, and it's
annoying to have SIGQUIT dump core on netbsd when it prints info on other
systems)
* in {recv,send}request(), factor a lot of duplicated code out into a
`cleanup' section at the end
* rework shell() a bit
 1.84 05-Oct-1999  lukem add TNFi copyright to all files i've done more than a minor amount of work to...
 1.83 05-Oct-1999  lukem enhancments from Marc Horowitz <marc@mit.edu> to improve connection timeouts:
* implement xsignal_restart(), which only sets the SA_RESTART flag if
specifically requested
* xsignal() is now a wrapper to xsignal_restart(). INFO, USR1, USR2 and WINCH
are restartable, ALRM, INT, PIPE and QUIT are not.
* improve getreply()'s timeout code to take advantage of the above.

other changes:
* improve wording of how globbing works for `classic' URLs (host:path).
suggested by John Refling <johnr@imageworks.com> in relation to PRs
[bin/8519] and [bin/8520]
* always compile in the `edit' command even if NO_EDITCOMPLETE defined.
it's just a no-op in the latter case, which is more consistent to
the users.
* always compile in about: support (i.e, remove NO_ABOUT).
i'm entitled to some vanity in this program...
* clean up some whitespace
 1.82 30-Sep-1999  lukem * In the !NI_NUMERICHOST case (i.e, getaddrinfo() challenged systems), portnum
should be in host order. found/fixed by Matthias Pfaller <leo@dachau.marco.de>
* parse_url(): improve checking of portnum, and add an extra argument to pass
back the parsed portnum to the caller (reduces a bit of code duplication)
* Move the KAME/WIDE copyrights after the BSD/TNFi ones. Since there was
significantly less code added under the former, it's only fair on the latter.
 1.81 30-Sep-1999  lukem * fix initialisation of home[]
* fetch_url(): if path would be NULL, return strdup("")
 1.80 29-Sep-1999  lukem * consistentnly use memset(a,0,c); there were some cases of memset(a,'\0',c)
* remove explicit extern int h_errno; it's in <netdb.h>
* add <termios.h> back to util.c; it contains struct winsize on some systems
 1.79 28-Sep-1999  lukem remove debugging cruft
 1.78 28-Sep-1999  lukem * add new commands:
lpage page local files
pdir as dir, but through your $PAGER
pls as ls, but through your $PAGER
* implement docase() (a la dotrans() et al) and use appropriately, rototilling
some duplicated code
* globulize(): modify to return a pointer to the strdup()ed result in all cases,
and hack the code that calls it to take this into account
* replace strcpy() and strncpy() with strlcpy()
* put(), getit(): use some aptly named local vars instead of argv[...]
* delint
 1.77 27-Sep-1999  lukem * idle(): rename to idlecmd(). certain linux distributions have an incompatible
prototype for idle() in <unistd.h> (which i thought was against namespace
and sensibility guidelines, but...)
* consistently use xsignal() instead of signal(). we get known behaviour
in all cases (SA_RESTART), which is good for some borken foreign systems.
* remove signal.h from most files; it's unnecessary now
* fetch_url(): use `long chunksize' instead of ssize_t; it's more portable, and
we're setting chunksize with strtol() anyway
* xsignal(): only use SA_RESTART if it exists. SunOS 4.x doesn't have it
but has the inverse (SA_INTERRUPT). the original function i was inspired
from had this support (lib/signal.c, W. Richard Stevens' `UNP 2nd ed Vol 1').
* remove <termios.h> from util.c; it should be unnecessary now
 1.76 26-Sep-1999  lukem * replace ifdefs against __SVR4 and __linux__ with DIRENT_MISSING_D_NAMLEN;
it's more portable and more obvious
* remove the mkgmtime() && HAVE_TIMEGM stuff:
a) why should netbsd have to define HAVE_TIMEGM to compile cleanly?
b) foreign compiles of ftp should just be linked with working
timegm function

a more portable version of this ftp client will be released as a 3rdparty
product; no use polluting our code with half-baked attempts...
 1.75 24-Sep-1999  lukem * use %lld instead of %qd to print out (long long) vars.
(slightly more portable; e.g, solaris supports this)
* remove some fluff (lint)
 1.74 24-Sep-1999  lukem fix a couple of thinkos in my recent work:
* abort_remote(): replace borken MIN(4,BUFSIZ) with just BUFSIZ; it
should have been MAX(4,BUFSIZ), but it's probably safe to assume that
BUFSIZ is at least 3... (fix from simonb)
* auth_url(): use the correct variable when calculating a buffer size.
 1.73 22-Sep-1999  lukem branches: 1.73.2;
* add support for `xferbuf', which sets both `sndbuf' and `rcvbuf'
* document the above three commands
* rototill the way the sndbuf and rcvbuf work. remove resetsockbufsize()
* use the appropriate socket buffer size as the size of the buffer that
the read()/write() loops use. speeds up things in some cases.
 1.72 22-Sep-1999  lukem replace snprintf() with strlcpy(), strlcat(), or direct assignment
where appropriate. (strlc*() are easier to port to foriegn systems).

XXX; there's still a few snprintf's in the progress meter stuff to convert
 1.71 21-Sep-1999  lukem * protect more of the AF_INET6 stuff with #ifdef INET6 (for portability)
* in the main data moving loops only call the initial gettimeofday() if
rate throttling is enabled (saves a system call per loop when not
throttling).
 1.70 21-Sep-1999  lukem add trailing . to message
 1.69 14-Sep-1999  mycroft warn()->warnx() in a couple of places.
 1.68 31-Aug-1999  christos The port number might have changed in the proxy case. Re-evaluate it.
 1.67 31-Aug-1999  christos fix proxy code. strtol() does not work very well for parsing port names.
 1.66 29-Aug-1999  christos make ftp work again with the traditional gethostbyname/getservbyname
interfaces.
 1.65 22-Aug-1999  lukem fetch_url() fixes:
- just display the hostname:port of the proxy url, rather than the full url.
this prevents someone `shoulder surfing' a proxy username/password
in $http_proxy. [suggested by perry]
- compact verbose notes for http fetchs; now displays
(via host:port, with authorization, with proxy authorization)
with each component being optional.
(and a couple introduced with the ipv6 mods...)
- don't override host with the canonical name; this prevented fetches from
http/1.1 virtual hosts from working if the virtual host was a CNAME.
[noted by bernd]
- call freeaddrinfo() if res was built with getaddrinfo()
 1.64 01-Aug-1999  lukem fix problem with www authentication: len passed to snprintf() for
auth_url() strings was 1 too small. noted by veego@
 1.63 12-Jul-1999  lukem * change fetch_ftp() to be fully rfc 1738 compliant; if the URL contains
the an empty directory (e.g, between `some' and `path' in
`ftp://host/some//path'), then execute `CWD ' (without a path).
This command will probably fail on rfc 959 compliant servers, so
issue a warning in this case and bail. [noted by cgd].
(i wonder if the people who wrote rfc 1738 actually realised that this
requirement appears to contravene the spec for `cwd' in rfc 959 ?)
* replace isurl() with isipv6addr(), and use appropriately. fixes
auto-login with `classic ftp URLs' (e.g, `ftp somehost:')
* cleanup and rework some of the ipv6 stuff in parse_url()
* prevent potential coredump in fetch_ftp() when parsing `;type=X'
* KNF a few lines
* fix a couple of comments
* cleanup the man page a bit
 1.62 06-Jul-1999  tron Make FTP downloads via HTTP proxy servers work again.
 1.61 04-Jul-1999  itojun fix 'junk pointer free' issue.
 1.60 02-Jul-1999  itojun add dual-stack (IPv4/v6) support. hope I broke no other part...
 1.59 29-Jun-1999  lukem [fear this; more ftp hacking from lukem :-]

features:
 1.58 27-Jun-1999  lukem don't search for trailing `;type=' in a NULL path...
fixes [bin/7800] by Mason Loring Bliss <mason@acheron.middleboro.ma.us>
 1.57 24-Jun-1999  christos include time.h explicitly, don't depend on other include side-effects
 1.56 20-Jun-1999  cgd split the "SMALL" #ifdefs into ones for NO_ABOUT, NO_EDITCOMPLETE, and
NO_PROGRESS. -DSMALL still implies all of those. progress meter support
isn't necessary for the smallest possible ftp client, but it adds very
little space and makes users' lives much better. Therefore, it should
be enabled for installation media if at all possible.
 1.55 02-Jun-1999  lukem * fix gate mode to login as `user@realhost' rather than using PASSERVE;
the latter only seemed to work for TIS Gauntlet and not TIS fwtk.
thanks to simonb@netbsd.org for testing this. fixes [bin/5556].
* if EOF (e.g, ^D) is entered at a username/password/account prompt which
happens to use fgets(), exit the login rather than treating EOF as CR.
* don't use the comma operator where separate statements are valid
* always use snprintf to copy stuff into malloced buffers, just in case
typos creep in and mean that the buffer ends up being overflowed
 1.54 12-May-1999  lukem parse http:// urls (and the $http_proxy variable) for [user:[pass]@]
elements, which are used for the initial authentication attempt (if
requested by the server). in the case of $http_proxy, use the values
for proxy authentication.
 1.53 28-Apr-1999  lukem * make parsing of ftp:// urls more RFC 1738 compliant;
- the path is split on `/', and each directory is CWD-ed into separately.
(from [standards/7484] by Alan Barrett <apb@iafrica.com>)
- support a trailing `;type=X' suffix, where X is a,i, or d. (d isn't
implemented, but it is recognised)
- the only non-compliant behaviour is that empty directories sections
(e.g `//') aren't run as `CWD ' - as a lot of ftpds don't like that.
Instead, treat this as a no-op.
* don't support globbing for ftp urls, since that's technically not
RFC compliant.
* fix a couple of man-page nits
 1.52 22-Mar-1999  lukem branches: 1.52.2;
* implement -R; restart non-proxied command-line FTP xfers
* fix fetch_ftp() so that hcode parsing is not done for file:// urls
(a } in the wrong place, and code at the wrong indent level...)
* change outfile to being a global (so it gets correctly reset)
* change parse_url to not remove leading '/' for non ftp urls.
whilst this is not totally rfc1738 compliant, other code kinda
assumes this is the case, and it doesn't hurt
 1.51 15-Mar-1999  christos Add a few more variables that end up in registers in gcc-2.8.1
 1.50 08-Mar-1999  lukem * add url_decode() - `in-place' decode %xx escapes in a given url component
* parse_url()
- only look for user[:pass] for an ftp url (per rfc1738)
- strip leading /'s in an ftp url. (almost per rfc 1738)
* fetch_url()
- decode a copy of the path and use that to build local filenames
- send port in http Host: header (suggested by cgd@netbsd.org)
* fetch_ftp()
- url_decode() the user, pass and path
- fix splitting of path into dir & file (partially from [bin/7073])
- don't bother caching the last host; it can cause problems when
using ftp:// transfers, or when the user changes between xfers
* improve documentation of auto-fetched url arguments (especially regarding
escape sequences in ftp:// urls)
* some whitespace & copyright updates

this should fix [bin/7073] William O Ferry <woferry@warp.wofme.com>,
as well as the metaissues raised in that PR.
 1.49 25-Jan-1999  lukem only print "Requesting..." messages if (verbose)
 1.48 24-Jan-1999  lukem * -v enables verbose & progress, -V disables both
* set setvbuf(ttyout, NULL, _IOLBF, 0) and remove a bunch of fflush(ttyout).
* use fwrite() instead of write() for progressmeter (don't intermix stdio
with non stdio ops)
 1.47 23-Jan-1999  lukem support -f, which forces a cache flush for http xfers using either
`Pragma: no-cache' (http/1.0) or `Cache-Control: no-cache' (http/1.1)
 1.46 05-Jan-1999  lukem add missing braces (this is C not python...). fixes coredump on solaris
 1.45 01-Jan-1999  lukem support http/1.1's `transfer-encoding: chunked' mode. [noted by hubertf & bad]
 1.44 01-Jan-1999  lukem * implement auth_url(); given a challenge, decode it, ask the user for
a response, and return the base64 encoded response. only the `Basic'
scheme is supported.
* implement base64_enc(), which base64 encodes the given buffer
* add support for http response codes 401 "authorization required" and 407
"proxy authorization required", using auth_url(). [requested by veego@]
 1.43 31-Dec-1998  lukem use uname(3) to determine info to send in User-agent: header. (suggested by cgd)
 1.42 29-Dec-1998  lukem * major code reorg; gut auto_fetch() and url_get() into a smaller auto_fetch(),
do_fetch() - which retrieves one file calling fetch_url() or fetch_ftp()
as necessary.
* don't http redirect more than 5 times for a given url
* send `User-Agent: NetBSD-ftp/1.4' header in http requests. (suggested
by Christoph Badura <bad@ora.de>)
* cleanup http return code parser, and add support for:
- 300 `Multiple Choices' - but only if the server returns a
preferred url in a Location: header because i'm *not* adding
a html parser to provide the user with options.
- 305 `Use Proxy [given in Location: header]'. (XXX: not tested)
* support http redirects to non-proxied ftp://urls. (bug discovered by
Chris Demetriou <cgd@netbsd.org>)
* auto-login to an ftp site (using the FTP protocol) if an ftp://host/dir/
style url is given and ftp_proxy is set. whilst this is less orthoganol
with other ftp://host/file urls it's *much* more convenient.
 1.41 27-Dec-1998  lukem be saner about the method to determine which $proxy to use
 1.40 22-Nov-1998  explorer Send an Accept: */* header, since some sites require content negotiation.
Grr.
 1.39 18-Nov-1998  itohy Temporary fix of UTC -> time_t conversion:
1. changed to use timegm(3) on NetBSD and
2. supply alternative code for other Unix-like OSs
(NetBSD ftp shall be portable for some reasons :-).

This fix closes PR #6448.

NOTE: This should be fixed again if a portable UTC to time_t
conversion method is specified in some standards.
 1.38 12-Nov-1998  lukem * fix previous in a more portable manner (timegm() is not portable/standard).
use mktime(), and adjust to GMT as per code in util.c::remotemodtime()
 1.37 12-Nov-1998  itohy HTTP uses UTC time, and changed to use timegm(3) instead of mktime(3)
for conversion of HTTP time.

XXX Unfortunately, timegm(3) is not portable (ex. SunOS 4 have, but 5 not).
 1.36 08-Oct-1998  lukem * hash & progress are mutually exclusive. [noted by mrg@netbsd.org]
* in autofetch mode, don't attempt to 'cd /' on first xfer (only on
subsequent xfers), as some ftp daemons don't permit that.
[noted by dbj@netbsd.org].
XXX: code could be smarter (but a lot more complicated to boot); this
works for now.
 1.35 01-Sep-1998  lukem fix handling of port on non-proxied auto-fetch ftp:// urls
 1.34 08-Aug-1998  lukem some compilers bitch about int used when enum expected...
 1.33 08-Aug-1998  lukem * fix skipping of leading / in dir, which may have resulted in
parsing junk memory (picked up whilst porting to solaris)
* remove superfluous var assignments
 1.32 08-Aug-1998  lukem pull in <sys/time.h> for utimes(), reset port to 0 in auto_fetch loop
 1.31 08-Aug-1998  lukem use utimes() instead of futimes() - the former is more portable
 1.30 08-Aug-1998  lukem technically, you need <sys/stat.h> for struct stat ....
 1.29 04-Aug-1998  lukem cleaner version of fvdl's previous fix, using xstrdup as well
 1.28 03-Aug-1998  fvdl Avoid using freed memory when using URLs and a proxy.
 1.27 03-Aug-1998  lukem features:
* support $no_proxy, which is a comma or space separated list of
host[:port] elements for which proxying is to be disabled.
(asked for by cgd in [bin/5027])
* if $FTPANONPASS is defined, use that as the anon ftp password
(instead of "`whoami`@")
* allow http URL's without a filename as long as an output file
is specified.

other stuff:
* implement parse_url(), which breaks up a URL into its bits, and use.
* simplify url_get() and auto_fetch() to use parse_url() and to not
modify the supplied URL or a copy of it.
* implement xmalloc() and xstrdup(); error-checked malloc()/strdup()
* add more consistency to messages, quoting strings in output as `%s'
 1.26 26-Jul-1998  lukem minor grammar fix
 1.25 26-Jul-1998  lukem * spell `retrieved' correctly!
* parse Last-Modified: HTTP headers for date, and change time of
retrieved file if parsing succeeded
* support file:/// and file://localhost/ URLs
* in url_get(), re-write byte moving code to consistently use fread()
and fwrite(), and check errors at end with ferror()
* add about:* easter-egg (#ifndef SMALL :-)
 1.24 22-Jul-1998  lukem * send 'Connection: close' in HTTP/1.1 headers, preventing the
remove server from using a persistent connection, which speeds
up such requests.
* support http 301 and 302 redirects
* rewrite guts of url_get() to use fparseln() et al instead of
read(s,&p,1)... enables each in the header to be parsed
as necessary
* rename login to ftp_login, to remove conflict with util.h::login
* cleanup verbose messages during http proxy requests
 1.23 10-Jul-1998  thorpej Add "sndbuf" and "rcvbuf" commands for setting the socket buffer sizes,
which in turn can allow the use of larger TCP windows. This is a work in
progress; there is not yet support for specifying global defaults or
user prefrences on a host/network basis.
 1.22 04-Jun-1998  lukem some fixes & enhancements from openbsd's ftp, with extra fixes by me:
* default to passive with active fallback. $FTPMODE modifies this behaviour.
-A forces active connection.
* support '-o outfile' for auto-fetched files. outfile can be a file,
`-' (for stdout), or '|command' (to output each file through command).
* support '-r waittime', which retries the connection after waittime seconds
if it fails.
* fix 'page file' when restart is non-zero.
* try all ip-addresses of a host in a http fetch (as the normal ftp fetch
does).

XXX: a ``broken pipe'' error sometimes occurs with -o '|command';
i haven't tracked this down yet.
 1.21 03-Jun-1998  tv Add the ability to work with HTTP-1.1-style virtual hosts, using the
HTTP/1.0 backwards compatible "Host:" field method.
 1.20 20-May-1998  christos - cast arg to is*() to unsigned char.
 1.19 18-Jan-1998  lukem define MAX_IN_PORT_T and use instead of USHRT_MAX. suggested by cgd@netbsd.org
 1.18 18-Jan-1998  lukem * ensure buffer for username is initialised, so ^D on username prompt
doesn't use garbage for the username. from "Soren S. Jorvang" <soren@t.dk>
in [bin/4559]
* use in_port_t for ports, and USHRT_MAX instead of 0xffff
(from millert@openbsd.org)
* use `NULL' instead of `(.... *)0' where appropriate.
 1.17 01-Nov-1997  lukem * in recvrequest(), ignore restart_point unless "RETR"ieving. fixes problems
where a remote completion or `mget' would confuse the client a `restart'
had been issued beforehand. now, `restart' is remembered until an operation
that can actually use it is invoked.
* in sendrequest(), don't reset restart_point upon entry. fixes `restart'
for `put' operations.
* if `restart' is invoked with no arguments, print current setting instead
of displaying a usage
* consistently use printf("%qd", (long long)restart_point) when displaying
restart_point
* use strto[lq]() instead of atol() when parsing `mark' and `restart' values
* remove unnecessary strlen()s when result of previous snprintf() will do
* replace a few malloc()/strcpy()s with strdup()s
* use SECSPERHOUR instead of '3600'
 1.16 21-Sep-1997  lukem branches: 1.16.2;
More cleanups from Todd Miller <Todd.Miller@courtesan.com>:
* Use an int, not "union wait".
* Move the "parsed_url" label so that the next statement is not an "else" --
some ansi C compilers don't like it the old way (SGI's for example).
* Deal with the possibility of getlogin(2) failing.
* Don't error out if the remote server doesn't support the "MDTM" command.
 1.15 13-Sep-1997  lukem Fixes from Todd Miller <Todd.Miller@courtesan.com>:
* use size_t instead of int in places
* use symbolic constants when using access()
 1.14 18-Aug-1997  lukem bugs fixed:
* don't interpret '-' or '|' when a local filename is determined from
the remote name (i.e, in mget, and in get with only one argument).
This is implemented using an extra argument to recvrequest().
Fixes a major security hole.
* clean up memory leak when using globulize()
* clean up a couple of comments
* fix wording in TNF copyright

features added:
* support for TIS fwtk gate-ftp servers:
* read defaults from $FTPSERVER && $FTPSERVERPORT
* start in gate-ftp mode if invoked as 'gate-ftp'
* toggle or set with 'gate [host [port]]'
 1.13 20-Jul-1997  lukem Send \r with \n in http requests. Allow "content-length" to be missing.
From [bin/3891] by Krister Walfridsson <cato@ulysses.df.lth.se>
 1.12 20-Jul-1997  lukem * use RCSID() && COPYRIGHT() macros
* cleanup for WARNS=1 (including some ugly '(void)&var' bits wrapped in
#ifdef __GNUC__ to shut up gcc warnings WRT setjmp/longjmp)
* use strtol() instead of atoi(), and more extensively check result of
conversion
* use u_int16_t instead of short or int for TCP port addresses
 1.11 29-Jun-1997  lukem Provide a different error message than `invalid url' when an
auto-login ftp URL is used when $ftp_proxy is defined. It now prints:
Auto-login using ftp URLs isn't supported when using $ftp_proxy
Should solve rest of [bin/3643].

Whilst this is inconsistant with the behaviour when $ftp_proxy isn't
defined, the following constraints apply:
* it's not possible to support ftp URL auto-login when $ftp_proxy is
defined, since it uses http not ftp, and you can't `login' to http
servers; fudging this would require a major rewrite of ftp anyway)
* silently ignoring $ftp_proxy and not using it if an ftp auto-login
URL is given is bad user interface design)
* mrg & others will harrass me if I remove support for autologin ftp URLs
when $ftp_proxy isn't defined, even though it made the behaviour
consistant whether $ftp_proxy was set or not.
 1.10 23-May-1997  lukem url_get(): use origline in some messages, simplifying them, and free up copy
of origline before returning. also, previous commit (fix for [bin/3643]) was
inspired by suggested fix in said PR by Alan Barrett <apb@iafrica.com>.
 1.9 23-May-1997  lukem Actually report why certain URLs are invalid, rather than just exit
with a non-zero exit code. fixes most of [bin/3643].
 1.8 21-Apr-1997  lukem * cleanup parser of ftp://[user:pass@]host[:port]/[dir/][file]. should
be more robust now. fixes [bin/3520]. (missed this in previous commit msg)
* rename bad_url: to bad_ftp_url:
 1.7 21-Apr-1997  lukem i
 1.6 14-Apr-1997  lukem More enhancements/bugfixes (when will it end?)
* differentiate between being connected, and being logged in
* cleanup some text messages
* support username & password ftp URLs (ftp://user:pass@host/) in non-proxy
situations; assume proxy supports it for proxy situations.
* cd to / before performing any autofetch transfers
* use strncasecmp in URL parsing. fix from <Todd.Miller@courtesan.com>
 1.5 05-Apr-1997  lukem * support $ftp_proxy for ftp:// transfers [bin/3245]
* add "more" & "less" as synonyms for "page"
* move editline setup code into controlediting(), and call appropriately.
only setup setup terminal if going into interactive mode. inspired
by Todd Miller <todd.miller@courtesan.com>
 1.4 16-Mar-1997  lukem Fixes from <Todd.Miller@cs.colorado.edu>, with some cleanup and reworks by me:
* only echo "Passive mode" in verbose mode; scripts that use ftp
may get unwanted output otherwise
* disable progress bar and modification time preservation when
retreiving to a non-regular files. fixes progress bar getting in
way of "get file /dev/tty"
* setup el_init() et al if editing is set, not if fromatty.
TODO: migrate this to a function, and call if editing is turned on later
in the session. also implement edit_cleanup if editing is turned off
* call el_set() after setting SIGWINCH handler. This fixes the problem
when suspending in a non-cbreak shell (e.g, csh) would trash your tty mode.
* reset interactive mode correctly in auto_fetch() mget mode
 1.3 13-Mar-1997  lukem Features:
* support remglobbing of auto_fetch arguments
* new flag - '-e'; disable editing
* "page file" == "get file |${PAGER-less}"

Bugfixes/cleanup:
* consistently use a trailing '.' on messages
* code cleanup, including buffer overrun fixes, use puts
and putchar in places, etc (inspired by OpenBSD mods)
* disable progress bar when local-file is a pipe or '-'
* skip \r in http headers
* fix remote ftpd slash bug more elegantly (so it works with ////)
* abort_remote(): check if cout==NULL before using it. should fix [bin/3273]
* fixed up cosmetic problems when complete_remote() generated errors from the
remote server (such as "no files found", "login with user and pass", ...)
done by adding extra argument to remglob(), which is a pointer to an error
buffer to put messages in rather than printing to stdout.
 1.2 01-Feb-1997  lukem [Yet Another Huge Ftp Commit - hopefully the last for a while,
barring any more little things people want added ...]

New features:
* progressmeter is now asynchronous, so "stalled" transfers can be
detected. "- stalled -" is displayed instead of the ETA in this case.
When the xfer resumes, the time that the xfer was stalled for is
factored out of the ETA. It is debatable whether this is better than
not factoring it out, but I like it this way (I.e, if it stalls for 8
seconds and the ETA was 30 seconds, when it resumes the ETA will still
be 30 seconds).
* verbosity can be disabled on the command line (-V), so that in auto-fetch
mode the only lines displayed will be a description of the file, and
the progress bar (if possible)
* if the screen is resized (and detected via the SIGWINCH signal), the
progress bar will rescale automatically.

Bugs fixed:
* progress bar will not use the last character on the line, as this can
cause problems on some terminals
* screen dimensions (via ioctl(TIOCWINSZ)) should use stdout not stdin
* progressmeter() used some vars before initialising them
* ^D will quit now. [fixes bin/3162]
* use hstrerror() to generate error message for host name lookup failure.
* use getcwd instead of getwd (it should have been OK, but why tempt fate?)
* auto-fetch transfers will always return a positive exit value upon failure
or interruption, relative to the file's position in argv[].
* remote completion of / will work, without putting a leading "///".
This is actually a bug in ftpd(1), where "NLST /" prefixes all names
with "//", but fixing every ftpd(1) is not an option...
 1.1 19-Jan-1997  lukem New features:
* Command line editing via editline(3) library.
* Context sensitive command and file completion, including remote files.

Enhancements to auto-fetch feature:
* Support for http:// URLs using the http protocol, including proxy HTTP
support via $htty_proxy if it's defined.
* The connection is kept open between successive files on the same host.
(obviously, this does not count for http requests.)
* Return value of ftp is 0 on no error, or the offset in argv[] of the file
which failed (i.e., argv[x] failed, ftp returns x).
* If the path in an ftp URL or classic format line has a trailing '/',
cd to the path and enter interactive mode. Fixes [bin/3011], albiet
requiring the user to help ftp in determining the operation.

Other changes:
* '-P port' works for normal ftp, and is the default for all classic style
auto-fetch transfers and for ftp URLs that don't specify the port.
(previously it would just work for the first xfer.)
* Some code moved into separated files along logical divisions.
* Editing and completion can be compiled out with -DSMALLFTP.
 1.16.2.3 23-Nov-1998  cgd pull up revs 1.37-1.40 from trunk (itohy)
 1.16.2.2 10-Nov-1998  cgd pull up rev(s) 1.18-1.36 from trunk. (feyrer)
 1.16.2.1 18-Nov-1997  mellon Pull rev 1.17 up from trunk (lukem)
 1.52.2.3 02-Jul-1999  perry pullup 1.57->1.58 (lukem)
 1.52.2.2 25-Jun-1999  cgd pull up rev 1.56 from trunk. (cgd)
 1.52.2.1 22-Jun-1999  perry pullup 1.52->1.55 (lukem)
 1.73.2.1 27-Dec-1999  wrstuden Pull up to last week's -current.
 1.112.2.1 23-Jun-2000  minoura Sync w/ netbsd-1-5-base.
 1.116.2.1 12-Nov-2001  he Pull up revision 1.127 (requested by tacha):
If in a proxied environment and no_proxy is true, use fetch_ftp()
to fetch FTP type URLs.
 1.155.2.9 28-Aug-2005  tron Pull up following revision(s) (requested by lukem in ticket #715):
usr.bin/ftp/version.h: revision 1.57
usr.bin/ftp/fetch.c: revision 1.164
Don't clear the trailing character on the auth_url() username;
we now use getline() and that newline strips for us.
Problem found & fixed by Mark Davies.
 1.155.2.8 24-Jul-2005  tron Pull up revision 1.163 (requested by lukem in ticket #606):
Add NO_USAGE and NO_DEBUG so that we can fit in the floppies again.
 1.155.2.7 24-Jul-2005  tron Pull up revision 1.162 (requested by lukem in ticket #606):
Implement:
int getline(FILE *stream, char *buf, size_t buflen, const char **errormsg)
Read a line from the FILE stream into buf/buflen using fgets(), so up
to buflen-1 chars will be read and the result will be NUL terminated.
If the line has a trailing newline it will be removed.
If the line is too long, excess characters will be read until
newline/EOF/error.
Various -ve return values indicate different errors, and errormsg
will be changed to an error description if it's not NULL.
Convert to use getline() instead of fgets() whenever reading user input
to ensure that an overly long input line doesn't leave excess characters
for the next input operation to accidentally use as input.
Zero out the password & account after we've finished with it.
Consistently use getpass(3) (i.e, character echo suppressed) when
reading the account data. For some reason, historically the "login"
code suppressed echo for Account: yet the "user" command did not!
Display the hostname in the "getaddrinfo failed" warning.
Appease some -Wcast-qual warnings. Fixing all of these requires
significant code refactoring. (mmm, legacy code).
 1.155.2.6 24-Jul-2005  tron Pull up revision 1.161 (requested by lukem in ticket #606):
* Only print the "Trying <address>..." message if verbose and
there's more than one struct addrinfo in the getaddrinfo() result.
* Don't use non-standard "u_int".
 1.155.2.5 24-Jul-2005  tron Pull up revision 1.160 (requested by lukem in ticket #606):
* fetch_ftp(): preserve 'anonftp' across a disconnect() so that multiple
ftp auto-fetches on the same command line login automatically.
* auto_fetch(): use an initialized volatile int to appease IRIX cc.
 1.155.2.4 24-Jul-2005  tron Pull up revision 1.159 (requested by lukem in ticket #606):
Use size_t instead of int where appropriate.
 1.155.2.3 24-Jul-2005  tron Pull up revision 1.158 (requested by lukem in ticket #606):
Fix some cast issues highlighted by Scott Reynolds using gcc 4 on OSX.4
 1.155.2.2 09-May-2005  tron Pull up revision 1.157 (requested by lukem in ticket #266):
gratuitous whitespace cleanup (before someone else jumps the gun...)
 1.155.2.1 09-May-2005  tron Pull up revision 1.156 (requested by lukem in ticket #264):
In fetch_url(), don't call freeaddrinfo(res0) too early, as we use pointers
to its contents later in the function.
Problem found by Onno van der Linden.
 1.182.2.1 09-Jan-2008  matt sync with HEAD
 1.183.6.1 18-May-2008  yamt sync with head.
 1.185.14.1 27-Oct-2014  msaitoh Pull up following revision(s) (requested by jmcneill in ticket #1928):
usr.bin/ftp/fetch.c: revision 1.206 via patch
don't pay attention to special characters if they don't come from the command
line (from jmcneill)
 1.185.8.1 13-May-2009  jym Sync with HEAD.

Third (and last) commit. See http://mail-index.netbsd.org/source-changes/2009/05/13/msg221222.html
 1.185.6.3 27-Oct-2014  msaitoh Pull up following revision(s) (requested by jmcneill in ticket #1928):
usr.bin/ftp/fetch.c: revision 1.206 via patch
don't pay attention to special characters if they don't come from the command
line (from jmcneill)
 1.185.6.2 20-Nov-2010  riz branches: 1.185.6.2.4;
Pull up following revision(s) (requested by lukem in ticket #1370):
usr.bin/ftp/extern.h: revision 1.78
usr.bin/ftp/fetch.c: revision 1.192
usr.bin/ftp/util.c: revision 1.153
Parse HTTP 'Date' entries in the `C' locale rather than the user's.
Fix from [bin/42917] (with minor changes), from KAMADA Ken'ichi.
 1.185.6.1 20-Nov-2010  riz Pull up following revision(s) (requested by lukem in ticket #1369):
usr.bin/ftp/fetch.c: revision 1.189
avoid NULL dereference in log output if the command line parser
failed to extract a port number from the URL
 1.185.6.2.4.1 27-Oct-2014  msaitoh Pull up following revision(s) (requested by jmcneill in ticket #1928):
usr.bin/ftp/fetch.c: revision 1.206 via patch
don't pay attention to special characters if they don't come from the command
line (from jmcneill)
 1.194.2.4 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.194.2.3 23-Jan-2013  yamt sync with head
 1.194.2.2 30-Oct-2012  yamt sync with head
 1.194.2.1 17-Apr-2012  yamt sync with head
 1.195.8.1 27-Oct-2014  snj Pull up following revision(s) (requested by christos in ticket #1175):
usr.bin/ftp/fetch.c: revision 1.206 via patch
don't pay attention to special characters if they don't come from the command
line (from jmcneill)
 1.195.6.1 27-Oct-2014  snj Pull up following revision(s) (requested by christos in ticket #1175):
usr.bin/ftp/fetch.c: revision 1.206 via patch
don't pay attention to special characters if they don't come from the command
line (from jmcneill)
 1.195.2.6 06-Jul-2017  snj Pull up following revision(s) (requested by nonaka in ticket #1428):
usr.bin/ftp/fetch.c: revision 1.227
Use the first name we requested the http/https URL for, not any name we ended
up with after random redirects.
 1.195.2.5 06-Jul-2017  snj Pull up following revision(s) (requested by nonaka in ticket #1423):
usr.bin/ftp/fetch.c: revision 1.226
handle proxy authentication correctly.
 1.195.2.4 16-Nov-2016  snj Pull up following revision(s) (requested by nonaka in ticket #1414):
usr.bin/ftp/fetch.c: revision 1.225
PR/51558: ast@: ftp dumps core after usage message when IPv6 URL lacks a slash.
Initialize variable so that we don't get random behavior on cleanup.
 1.195.2.3 27-Aug-2016  bouyer Apply patch, requested by nonaka in ticket #1375:
src/usr.bin/ftp/cmds.c: patch
src/usr.bin/ftp/fetch.c: patch
src/usr.bin/ftp/ftp.1: patch
src/usr.bin/ftp/ftp.c: patch
src/usr.bin/ftp/ftp_var.h: patch
src/usr.bin/ftp/main.c: patch
src/usr.bin/ftp/ssl.c: patch
src/usr.bin/ftp/ssl.h: patch
src/usr.bin/ftp/version.h: patch
Update ftp(1) to version 20150912, adding https via proxy support.
 1.195.2.2 27-Oct-2014  snj Pull up following revision(s) (requested by christos in ticket #1175):
usr.bin/ftp/fetch.c: revision 1.206 via patch
don't pay attention to special characters if they don't come from the command
line (from jmcneill)
 1.195.2.1 17-Dec-2013  bouyer Apply patch, requested by tron in ticket #997:
usr.bin/ftp/Makefile patch
usr.bin/ftp/cmds.c patch
usr.bin/ftp/cmdtab.c patch
usr.bin/ftp/extern.h patch
usr.bin/ftp/fetch.c patch
usr.bin/ftp/ftp.1 patch
usr.bin/ftp/ftp.c patch
usr.bin/ftp/ftp_var.h patch
usr.bin/ftp/main.c patch
usr.bin/ftp/progressbar.c patch
usr.bin/ftp/ssl.c patch
usr.bin/ftp/ssl.h patch
usr.bin/ftp/util.c patch
usr.bin/ftp/version.h patch

Add HTTPS support to ftp(1).
 1.198.2.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.198.2.1 25-Feb-2013  tls resync with head
 1.205.4.7 18-Mar-2017  snj Pull up following revision(s) (requested by nonaka in ticket #1353):
usr.bin/ftp/fetch.c: revision 1.227
Use the first name we requested the http/https URL for, not any name we ended
up with after random redirects.
 1.205.4.6 18-Dec-2016  snj Pull up following revision(s) (requested by nonaka in ticket #1331):
usr.bin/ftp/fetch.c: revision 1.226
handle proxy authentication correctly.
 1.205.4.5 01-Nov-2016  snj Pull up following revision(s) (requested by dholland in ticket #1265):
usr.bin/ftp/fetch.c: revision 1.225
PR/51558: ast@: ftp dumps core after usage message when IPv6 URL lacks a slash.
Initialize variable so that we don't get random behavior on cleanup.
 1.205.4.4 29-Apr-2016  snj branches: 1.205.4.4.2;
Pull up following revision(s) (requested by nonaka in ticket #1153):
usr.bin/ftp/fetch.c: revision 1.223
PR/51043: Yorick Hardy: ftp(1) should use the port number for CONNECT
 1.205.4.3 13-Mar-2016  martin Pull up following revision(s) (requested by nonakap in ticket #1133):
usr.bin/ftp/fetch.c: revision 1.208-1.221
usr.bin/ftp/cmds.c: revision 1.136-1.137
usr.bin/ftp/ssl.c: revision 1.5
usr.bin/ftp/ftp.c: revision 1.165-1.166
usr.bin/ftp/ftp_var.h: revision 1.84

Workaround const issues of SSL_set_tlsext_host_name.

Use the proper format "[IPv6 address]:port" when reporting connection
attempts to IPv6 endpoints.

(Hopefully) fix build without IPv6 support

Try to factor out some code, this is completely out of control.

Separate no_proxy handling.

Factor the proxy handling code out.

Fix compile failure without WITH_SSL.

PR/50438: NONAKA Kimihiro: ftp(1): CONNECT method support

make DPRINTF/DWARN always statements.

Fix to connect https via proxy.

Fix ttyout message.

Simplify and factor out connect message

Split the position/size parsing into a separate function.

Mark function as only needed with ssl.

Fix downloads of local files using file:// URLs

Initialize the token match pointer.

use sizeof() and array notation.
CID 1354295: Array overrun.
 1.205.4.2 05-Nov-2015  riz Pull up following revision(s) (requested by wiz in ticket #981):
usr.bin/ftp/ftp.1: revision 1.135
usr.bin/ftp/ssl.c: revision 1.3
usr.bin/ftp/ssl.c: revision 1.4
usr.bin/ftp/ssl.h: revision 1.3
usr.bin/ftp/version.h: revision 1.86
usr.bin/ftp/version.h: revision 1.87
usr.bin/ftp/fetch.c: revision 1.207
usr.bin/ftp/main.c: revision 1.123
Add -x xferbufsize to set xferbuf size.
Implement -x xferbufsize set the socket send and receive buffer size,
as per 'xferbuf' in interactive mode.
Patch from Nicholas Mills (via private mail), with minor adjustment by me.
Add Server Name Indication (SNI) support for https.
Needed for e.g. some github URLs.
Bump version for SNI support.
servername cannot be NULL here.
Noted by joerg@.
 1.205.4.1 26-Oct-2014  martin Pull up following revision(s) (requested by christos in ticket #158):
usr.bin/ftp/fetch.c: revision 1.206
don't pay attention to special characters if they don't come from the command
line (from jmcneill)
 1.205.4.4.2.1 18-Jan-2017  skrll Sync with netbsd-5
 1.223.2.4 20-Mar-2017  pgoyette Sync with HEAD
 1.223.2.3 07-Jan-2017  pgoyette Sync with HEAD. (Note that most of these changes are simply $NetBSD$
tag issues.)
 1.223.2.2 04-Nov-2016  pgoyette Sync with HEAD
 1.223.2.1 06-Aug-2016  pgoyette Sync with HEAD
 1.226.2.1 21-Apr-2017  bouyer Sync with HEAD
 1.228.4.3 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1763:

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/cmds.c up to 1.141
usr.bin/ftp/complete.c up to 1.47
usr.bin/ftp/domacro.c up to 1.23
usr.bin/ftp/extern.h up to 1.82
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp.c up to 1.174
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/progressbar.c up to 1.24
usr.bin/ftp/progressbar.h up to 1.9
usr.bin/ftp/ssl.c up to 1.12
usr.bin/ftp/ssl.h up to 1.5
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

ftp(1): validate address from PASV and LPSV response.
ftp(1): use raw write(2) instead of fwrite(3) to avoid stream
corruption because of the progress bar interrupts.
Fixes for PR 56219 and PR 55857.
PR 57003: Support relative redirects.
 1.228.4.2 12-Sep-2022  martin Backout ticket #1763 for now - trust anchors are not solved.
 1.228.4.1 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1763:

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/cmds.c up to 1.141
usr.bin/ftp/complete.c up to 1.47
usr.bin/ftp/domacro.c up to 1.23
usr.bin/ftp/extern.h up to 1.82
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp.c up to 1.174
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/progressbar.c up to 1.24
usr.bin/ftp/progressbar.h up to 1.9
usr.bin/ftp/ssl.c up to 1.11
usr.bin/ftp/ssl.h up to 1.5
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

ftp(1): validate address from PASV and LPSV response.
ftp(1): use raw write(2) instead of fwrite(3) to avoid stream
corruption because of the progress bar interrupts.
Fixes for PR 56219 and PR 55857.
PR 57003: Support relative redirects.
 1.230.4.1 10-Jun-2019  christos Sync with HEAD
 1.231.2.5 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1523

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/ssl.c up to 1.12
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

PR 57003: Support relative redirects.
 1.231.2.4 12-Sep-2022  martin Back out ticket #1523 for now - trust anchor validation is not solved.
 1.231.2.3 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1523

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/ssl.c up to 1.11
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

PR 57003: Support relative redirects.
 1.231.2.2 24-Oct-2021  martin Pull up following revision(s) (requested by lukem in ticket #1365):

usr.bin/ftp/fetch.c: revision 1.233

Use raw write(2) instead of fwrite(3) to avoid stream corruption because
of the progress bar interrupts. From RVP.
 1.231.2.1 14-Jun-2021  martin Pull up following revision(s) (requested by lukem in ticket #1292):

usr.bin/ftp/fetch.c: revision 1.232

fetch_url: improve signal handler restoration

Use SIG_ERR not NULL as the indicator that a signal handler
hasn't been changed, so that SIG_DFL (equivalent to NULL)
will be restored.

Fix restoration of SIGQUIT; use the old handler not SIGPIPE's.
 1.235.2.4 02-Dec-2024  martin Pull up following revision(s) (requested by lukem in ticket #1021):

usr.bin/ftp/ftp.c: revision 1.178
usr.bin/ftp/version.h: revision 1.99
usr.bin/ftp/ruserpass.c: revision 1.35
usr.bin/ftp/main.c: revision 1.134
usr.bin/ftp/main.c: revision 1.135
usr.bin/ftp/progressbar.c: revision 1.27
usr.bin/ftp/util.c: revision 1.165
usr.bin/ftp/cmds.c: revision 1.144
usr.bin/ftp/extern.h: revision 1.84
usr.bin/ftp/fetch.c: revision 1.242
usr.bin/ftp/ftp.1: revision 1.160

s/bninary/binary/ in comment.

extract duplicate code into a function.

Check bounds when copying to destination.

Remove const where the const string ended up being overwritten.

use unsigned when doing shifts.

remove const when string gets overwritten.

ftp: exit non-zero if short http transfer when filesize is known
If a http file size is known and the fetch finishes with less bytes
transferred, exit non-zero.
Bump version to 20241129.
PR bin/54713
PR bin/58281

ftp: help improvements
Document -? as a separate mode.
Document -H HEADER in the usage.
Clarify units for -b and -x.
Consistent argument names between ftp -? and ftp(1).

ftp: order getopt Upper before lower
Consistently order options in getopt and the switch
with the upper case option before the lower case option.
This makes it easier to cross-reference with -? and ftp(1).
No functional change.
 1.235.2.3 13-Oct-2024  martin Pull up following revision(s) (requested by riastradh in ticket #970):

tests/usr.bin/Makefile: revision 1.42
usr.bin/ftp/ruserpass.c: revision 1.34
usr.bin/ftp/main.c: revision 1.130
usr.bin/ftp/ssl.c: revision 1.17
usr.bin/ftp/main.c: revision 1.131
usr.bin/ftp/ssl.c: revision 1.18
usr.bin/ftp/main.c: revision 1.132
usr.bin/ftp/ssl.c: revision 1.19
usr.bin/ftp/main.c: revision 1.133
distrib/sets/lists/tests/mi: revision 1.1342
usr.bin/ftp/ftp.1: revision 1.151
usr.bin/ftp/ftp.1: revision 1.152
usr.bin/ftp/progressbar.c: revision 1.25
usr.bin/ftp/ftp.1: revision 1.153
usr.bin/ftp/progressbar.c: revision 1.26
usr.bin/ftp/ftp.1: revision 1.155
usr.bin/ftp/ftp.1: revision 1.156
usr.bin/ftp/fetch.c: revision 1.239
usr.bin/ftp/ftp.1: revision 1.157
usr.bin/ftp/ftp.1: revision 1.158
usr.bin/ftp/ftp.1: revision 1.159
usr.bin/ftp/ftp_var.h: revision 1.87
etc/mtree/NetBSD.dist.tests: revision 1.208
usr.bin/ftp/ftp_var.h: revision 1.88
usr.bin/ftp/ftp_var.h: revision 1.89
usr.bin/ftp/cmds.c: revision 1.142
usr.bin/ftp/util.c: revision 1.168
usr.bin/ftp/cmds.c: revision 1.143
tests/usr.bin/ftp/custom_headers.sh: revision 1.1
usr.bin/ftp/ssl.c: revision 1.20
usr.bin/ftp/complete.c: revision 1.48
tests/usr.bin/ftp/Makefile: revision 1.1
tests/usr.bin/ftp/t_custom_headers.sh: revision 1.1
usr.bin/ftp/fetch.c: revision 1.240
usr.bin/ftp/fetch.c: revision 1.241
usr.bin/ftp/ftp.c: revision 1.176
usr.bin/ftp/ftp.c: revision 1.177
(all via patch)

ftp(1): wording and formatting improvements

Fix grammar issue with "Support values" reported in private mail.
Document all file transfer types in "type" and cross-reference that.
Consistency fixes in describing file transfer parameters and types.

Fix some mandoc -Tlint issues (except "useless macro: Tn").

Add -b <buflen> to specify the buffer size.

ftp: bump FTPBUFLEN from 4kB to 16kB
sourceforge.net returns a 5kB content-security-policy.
Analyzed by mlelstv@ who reports usual limits are between 4kB and 48kB.
default is now 16K

ftp: improve -b documentation

Order -b bufsize in the synopsis.

Document the actual default value.

ftp: improve units used in comments and errors
Use "KiB" instead of "K" in errors.
Clarify related comments.

pass some lint.

PR/58581: Sunil Nimmagadda: Add flag to allow specifying extra http header
fields.

ftp(1): Nix trailing whitespace in man page.
No functional change intended.

PR bin/58581: ftp(1) should allow specifying header fields in http requests
fix markup (h -> H), explain about multiple headers, fix usage (from RVP)

Don't forget the dot, use the intended macro name (I think),
and improve the wording a little. (All related to the -H option.)
ftp(1): Add test for custom HTTP header fields.

Based on a patch from Sunil Nimmagadda.

PR bin/58581: ftp(1) should allow specifying header fields in http
requests
 1.235.2.2 15-Jan-2024  martin Pull up following revision(s) (requested by mlelstv in ticket #555):

usr.bin/ftp/fetch.c: revision 1.237
usr.bin/ftp/fetch.c: revision 1.238

Fix HTTPS through Proxy.

While a regular HTTP Proxy, requires the absolute URL with protocol
and host part, yyou must only send the relative URL through a
CONNECT tunnel (you are talking to the target server).

Don't finish downloading an empty file with 'already done' before it is
created locally.
 1.235.2.1 16-May-2023  martin Pull up following revision(s) (requested by lukem in ticket #170):

usr.bin/ftp/ssl.c: revision 1.13
usr.bin/ftp/fetch.c: revision 1.236
usr.bin/ftp/util.c: revision 1.166
usr.bin/ftp/main.c: revision 1.129
usr.bin/ftp/extern.h: revision 1.83
usr.bin/ftp/ftp.1: revision 1.148
usr.bin/ftp/cmdtab.c: revision 1.53
usr.bin/ftp/version.h: revision 1.96

Add option sslnoverify to control validation of SSL certificates.

Add netrc processing to fetch-mode (URL on command line) to enable options and autologin
via netrc.

Fix SSL cleanup in some error paths.

Certificate validation is now enabled by default. Set FTPSSLNOVERIFY=1 in environment
or configure a corresponding init macro via netrc to not validate certs (required if
you haven't installed a required CA certificate for OpenSSL).

Discussed with lukem@ on icb.
 1.239.2.1 02-Aug-2025  perseant Sync with HEAD
 1.1 10-Nov-1998  cgd branches: 1.1.2;
file fparseln.c was initially added on branch netbsd-1-3.
 1.1.2.1 10-Nov-1998  cgd add new file (via patch) to back-port ftp to 1.3 branch. (feyrer)
 1.160 29-Nov-2024  lukem ftp: help improvements

Document -? as a separate mode.
Document -H HEADER in the usage.
Clarify units for -b and -x.
Consistent argument names between ftp -? and ftp(1).
 1.159 30-Sep-2024  kre Don't forget the dot, use the intended macro name (I think),
and improve the wording a little. (All related to the -H option.)
 1.158 30-Sep-2024  christos fix markup (h -> H), explain about multiple headers, fix usage (from RVP)
 1.157 27-Sep-2024  riastradh ftp(1): Nix trailing whitespace in man page.

No functional change intended.

PR bin/58581: ftp(1) should allow specifying header fields in http requests
 1.156 25-Sep-2024  christos PR/58581: Sunil Nimmagadda: Add flag to allow specifying extra http header
fields.
 1.155 19-Jul-2024  lukem ftp: improve -b documentation

Order -b bufsize in the synopsis.
Document the actual default value.
 1.154 17-Apr-2024  gutteridge branches: 1.154.2;
ftp.1: drop a sentence that's no longer accurate
 1.153 19-Feb-2024  christos default is now 16K
 1.152 18-Feb-2024  christos Add -b <buflen> to specify the buffer size.
 1.151 09-Dec-2023  lukem ftp(1): wording and formatting improvements

Fix grammar issue with "Support values" reported in private mail.
Document all file transfer types in "type" and cross-reference that.
Consistency fixes in describing file transfer parameters and types.
Fix some mandoc -Tlint issues (except "useless macro: Tn").
 1.150 25-Feb-2023  uwe ftp(1): better mark up for url vs cd example
 1.149 25-Feb-2023  uwe ftp(1): minor markup tweaks

Use .Ql instead of .Sq Li, add some missing ones. Use .Pq instead of
explicit () for longer phrases - these are easier to read in the
postscript output b/c of extra spacing.
 1.148 25-Feb-2023  mlelstv Add option sslnoverify to control validation of SSL certificates.
Add netrc processing to fetch-mode (URL on command line) to enable options and autologin
via netrc.
Fix SSL cleanup in some error paths.

Certificate validation is now enabled by default. Set FTPSSLNOVERIFY=1 in environment
or configure a corresponding init macro via netrc to not validate certs (required if
you haven't installed a required CA certificate for OpenSSL).

Discussed with lukem@ on icb.
 1.147 30-Aug-2022  christos branches: 1.147.2;
Add cert verification, together with an environment variable "NO_CERT_VERIFY",
to turn it off.
 1.146 25-Apr-2021  lukem ftp(1): consistently use FTP for protocol use.
 1.145 25-Apr-2021  lukem ftp(1): consistent Ic (not Nm) for commands
 1.144 31-Jan-2021  lukem ftp(1): more $https_proxy documentation

Document $https_proxy in ENVIRONMENT.
(It was already documented elsewhere).

Fixes PR bin/51883
 1.143 06-Jan-2021  lukem ftp(1): fix description of "debug"

"debug" command and documentation got accidentally renamed
to "ftp_debug" 13 years ago, and was only partially fixed.
 1.142 18-Jul-2020  lukem ftp: add -? for help. improve synopsis

Add -? to display usage synopsis and help to stdout.
This allows for "ftp -? | less", which is more user friendly.
Errors still show usage to stderr.
Consistency improvements in some usage text.
 1.141 15-Jul-2020  uwe Try to improve markup for better PostScript output.
 1.140 15-Jul-2020  uwe Do not use "[...]", just "..." is enough.

Conventionally the ellipsis already expresses optional repetition,
e.g. .Ar without arguments produces "file ...".
 1.139 15-Jul-2020  pgoyette Remove now-extraneous Op since we already have Oo and Oc to enclose
the Ar port.
 1.138 15-Jul-2020  lukem ftp.1: don't wrap "[[user@]host [port]]"
 1.137 13-Jul-2020  lukem ftp(1): consistency tweaks
 1.136 03-Jul-2017  wiz branches: 1.136.8;
Remove workaround for ancient HTML generation code.
 1.135 23-Apr-2015  lukem branches: 1.135.8;
Add -x xferbufsize to set xferbuf size.

Implement -x xferbufsize set the socket send and receive buffer size,
as per 'xferbuf' in interactive mode.

Patch from Nicholas Mills (via private mail), with minor adjustment by me.
 1.134 22-Dec-2012  christos branches: 1.134.8;
document https support, mentioning certificates are not validated, and obey
https_proxy.
 1.133 08-Apr-2012  wiz branches: 1.133.2;
Remove unnecessary Bk/Ek pairs from SYNOPSIS.
No effective change except where I used the opportunity to sort options
and/or option descriptions.
 1.132 22-Mar-2012  wiz Fix whitespace nits. Suggested by Bug Hunting.
 1.131 05-Mar-2010  lukem branches: 1.131.6; 1.131.8;
Back to using 'RFC xxxx' instead of 'RFCxxxx'
 1.130 11-Jul-2009  joerg Fix markup.
 1.129 04-May-2009  wiz New sentence, new line.
 1.128 12-Mar-2009  joerg Don't workaround ancient macro argument limit with .Xo/.Xc. Don't use
.Sm off/.Sm on.
 1.127 30-Sep-2008  lukem branches: 1.127.6;
update copyrights
 1.126 13-May-2008  wiz Bump date; punctuation improvements.
 1.125 10-May-2008  skd Add epsv6 and epsv to disable extended passive mode for ipv6 or both ipv4 and ipv6 respectively. This hack is due to our friends a Juniper Networks who break
epsv in ipv6. Should be fixed in ScreenOS 6.2.X.
 1.124 02-May-2008  martin branches: 1.124.2;
Move TNF licenses to 2 clause form
 1.123 08-Jan-2008  reed branches: 1.123.4;
Sort a command. It was was renamed in 1.113 but not resorted.
(I noticed when reading man page but didn't quickly find what I was
looking for.)
Also add a missing period.
 1.122 02-Dec-2007  wiz Sort options.
 1.121 20-Aug-2007  perry branches: 1.121.2;
Never use "utilize". It means exactly the same thing as "use", but it
is longer and more cumbersome. It can always be replaced with "use"
without any change in meaning.
 1.120 06-Aug-2007  lukem branches: 1.120.2;
reorder location of "-s srcaddr" in synopsis
 1.119 18-Jul-2007  lukem Document about:ftp and about:version.
 1.118 24-May-2007  lukem Display times in RFC2822 form rather than using ctime(3), since
the former is more explicit about the timezone offset.
 1.117 15-May-2007  lukem * Modify parse_url() to consistently strip the leading `/' off ftp URLs.
Fixes PR 17617.
* Use 'RFCnnnn' (with leading 0) instead of 'RFC nnnn', to be
consistent with the style in the RFC index.
* Refer to RFC3916 instead of 1738 or 2732.
* Expand the list of supported RFCs in ftp(1) to contain the document
name as well.
 1.116 10-May-2007  lukem Replace references from draft-ietf-ftpext-mlst-NN to RFC 3659.
 1.115 17-Apr-2007  lukem * Implement -s srcaddr; uses srcaddr as the local IP address for all
connections.
Based on code in the version of ftp that FreeBSD had before they
replaced it with lukemftp.
* Move error message handling into ftp_connect() rather than in the
caller, so that more specific error reporting can occur.
* Improve consistency of various warning and error messages.
 1.114 27-Oct-2006  lukem Use "see" instead of "c.f.".
Per feedback from Valeriy E. Ushakov.
 1.113 31-Jan-2006  christos rename debug to ftp_debug. grr libssh.
 1.112 06-Jul-2005  wiz In macdef, point to $ command (from jmc@openbsd), and split paragraph (it was getting a bit big).
 1.111 06-Jul-2005  wiz Use more macros, and use them more consistently.
Fix a few typos.
 1.110 05-Jul-2005  lukem Improve the description of the macros.
Patch from David H. Gutteridge in PR 24296.
 1.109 20-Feb-2005  wiz branches: 1.109.2;
If we mark up RFC, then avoid using "command modifier" for it.
 1.108 15-Jan-2005  lukem preempt wizd; crank date
 1.107 15-Jan-2005  lukem Expand description of http_proxy by suggesting the use of RFC 1738 '%xx'
encoding for "unsafe URL" characters in usernames and passwords.
 1.106 04-Jan-2005  lukem fix copyright year
 1.105 01-Dec-2004  wiz Fix typo.
 1.104 07-Oct-2004  wiz Drop trailing whitespace.
 1.103 07-Oct-2004  dan pre-empt wizd (bump date for previous.)
 1.102 07-Oct-2004  dan Note potentially surprising file-saving behaviour in case of HTTP redirects
 1.101 19-Dec-2003  lukem branches: 1.101.2;
-n is ignored for auto-fetch transfers
 1.100 07-Aug-2003  agc Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22365, verified by myself.
 1.99 31-Jul-2003  lukem * $FTPUSERAGENT overrides the HTTP User-Agent header.
Based on patch from Douwe Kiela.
* Add about:tnftp
* Fix URL in about:netbsd
* Crank version
 1.98 09-Jul-2003  lukem tweak previous; no need to have .Sm on and then just .Sm off
 1.97 30-Jun-2003  wiz Fixes to make these man page look nice with groff-1.19.
 1.96 26-Jun-2003  wiz Quote punctuation so macro works on it (another one).
 1.95 26-Jun-2003  wiz Quote punctuation so macro works on it.
 1.94 14-May-2003  wiz Uppercase URL.
 1.93 28-Feb-2003  lukem crank dates
 1.92 25-Feb-2003  wiz .Nm does not need a dummy argument ("") before punctuation or
for correct formatting of the SYNOPSIS any longer.
 1.91 14-Feb-2003  grant The correct capitalisation of 'NetBSD.org' is (you
guessed it) 'NetBSD.org'.

some mdoc fixes.
 1.90 20-Jan-2003  wiz Bump date for last.
 1.89 20-Jan-2003  toddpw Add a "macdef" example and a short section explaining that you may need to
use it to disable extended passive mode when your firewall is obnoxious.
This did in fact solve a problem I was having with a firewall at work, and
the status message was taken directly from a problem session fixed by this.
 1.88 02-Oct-2002  wiz Typo and Americanism. From Adrian Mrva.
 1.87 30-Sep-2002  grant New sentence, new line.
 1.86 27-Aug-2002  christos Add a -q <quit-time> flag to abort a transfer if it has stalled for <quit-time>
seconds. Ok'd by luke.
 1.85 07-Jul-2002  lukem Various improvements in wording and marking,
from Alan Barrett <apb@cequrux.com>, with a bit of tweaking from me.
 1.84 04-Jul-2002  lukem highlight that for ftp:// auto-fetches, read access is required on
intermediate directories because ftp(1) obeys RFC1738. for [bin/15419]
 1.83 05-Jun-2002  lukem - when showing the final progress bar, replace "00:00 ETA" with the
elapsed time. (suggested by simonb)
- actually display transfer stats after a URL fetch. (bug introduced a
*long* time ago)
- update copyright & version
 1.82 18-May-2002  lukem branches: 1.82.2;
Fix description for "form", "mode", and "struct" commands.
Inspired by [bin/16736] from Steve McClellan <steve.mcclellan@radisys.com>
 1.81 08-Feb-2002  ross Generate <>& symbolically. I'm avoiding .../dist/... directories for now.
 1.80 26-Dec-2001  lukem update copyrights
 1.79 23-Dec-2001  lukem Add -4 to force IPv4 and -6 to force IPv6 address usage.
From Hajimu UMEMOTO, via Mike Heffner of FreeBSD.

(FreeBSD has imported NetBSD's ftp as their ftp client;
Mike is sending back some of their local changes).
 1.78 01-Dec-2001  wiz Punctuation nits.
 1.77 01-Dec-2001  wiz Whitespace cleanup.
 1.76 20-Jun-2001  lukem put "site" in alphabetical order. noted by Mike Barcroft in private email
 1.75 27-Nov-2000  lukem be more explicit that $ftp_proxy and $http_proxy are not supported for
interactive sessions
 1.74 15-Nov-2000  lukem - implement "mreget"; as per "mget" but uses "reget" instead of "get"
- add -N netrc and $NETRC, as methods to select an alternative .netrc file
- cache local user name and home directory for further use
- in mget(), use docase() instead of a local version to do the case
conversion.
 1.73 28-Sep-2000  lukem clarify that $ftp_proxy only works for full URLs and can't be used for
interactive connections.
 1.72 28-Jul-2000  lukem rename "opts" to "remopts", so people used to "o host" don't get bitten...
 1.71 18-Jul-2000  lukem add rfc 2389, since that's now supported
 1.70 18-Jul-2000  lukem add support for FEAT and OPTS commands with `features' and `opts'.
(from RFC 2389).

add support for MLST & MLSD (machine parseble listings) with 'mlst', 'mlsd'
and 'pmlsd' (mlsd |$PAGER) commands. (from draft-ietf-ftpext-mlst-11)

rename remotesyst() to getremoteinfo(), and modify to parse the result from
FEAT (if supported), and take into account the support for the various
extensions such as MDTM, SIZE, REST (STREAM), MLSD, and FEAT/OPTS.
put each feature into one of the following categories:
- known to work (explicit FEAT)
- unknown but assume works until explicit failure, when it's
then tagged as `known not to work'.
- known not to work (FEAT succeeded but didn't return anything,
or was unknown and then explicit failure)
assign results into features[] matrix.

add support to getreply() so that an optional callback will be called
for each line received from the server except for the first and last.
this is used in FEAT (and MLST) parsing.

modify various commands to check if REST (STREAM), MDTM and SIZE are
explicitly or implicitly supported before using.

fix `syst' when verbose is off.

minor knf (indent goto labels by one space, etc).

simply various command usage handlers by assuming that argv != NULL except
for quit() and disconnect().
 1.69 05-Jun-2000  lukem - fix ai_unmapped() to be a no-op in the !def INET6 case
- display `(-INET6)' at the end of the version string if !def INET6
- clarify in the man page that IPv6 support may not be present (for lukemftp :)
 1.68 01-Jun-2000  itojun updated comment on IPv4 mapped address. sync with kame.
 1.67 31-May-2000  lukem Add support for 'ftp -u url file ...', to upload a list of files to given url.
Mostly based on [bin/10019] by Scott Aaron Bamford <sab@ansic.net>
 1.66 31-May-2000  lukem Fix examples on using pipes in local filenames. AFAICT, ftp has always
required `dir . |more' not as `dir |more' treats `|more' as the remote
filename. Resolves [bin/9922] by Geoff Wing <mason@primenet.com.au>
 1.65 30-May-2000  itojun document IPv4 mapped address twists.
- ftp(1): treats IPv4 mapped destination as IPv4 peer, not native IPv6 peer.
this does not support network with SIIT translator.
- rshd(8)/rlogind(8): rejects accesses from IPv4 mapped peer, to avoid
possible abuse of IPv4 mapped addr (rshd/rlogind use source address-based
auth so it is important to check the condition).
 1.64 28-May-2000  lukem Change `ls' to use the `LIST' and not `NLST' FTP protocol command.
Now that after many years on not caring we find certain popular
ftp servers are starting to obey RFC959 to the letter of the law
and will only return a list of filenames (not directories or
other filetypes) in the output of `NLST', then `LIST' is more useful
in this case. (Note that the aforementioned pedanticness means that
filename completion isn't as useful as it could be...)
Fixes [bin/8937] by David A. Gatwood <dgatwood@deepspace.mklinux.org>
 1.63 01-May-2000  lukem branches: 1.63.2;
convert to ANSI KNF
 1.62 01-May-2000  lukem * Add support for `fget localfile', which reads a list of filenames to
retrieve from localfile. Based on work by Darren Reed.
* Crank version.
* Update copyright dates.
 1.61 21-Dec-1999  lukem change references from draft-ietf-ipngwg-url-literal-01.txt to RFC 2732
 1.60 26-Nov-1999  lukem * complete_remote(): use remglob("", ...) instead of remglob(".", ...),
for listings of the current working directory; some ftp servers don't
like `NLST .'.
[noted by Giles Lean <giles@nemeton.com.au>]
* recvrequest(): treat remote=="" as remote==NULL when calling command().
(to support the above change)
* support `[user@]' in `[user@]host' and `[user@]host[:][path]'.
[based on idea (and initial code) from David Maxwell <david@fundy.ca>]
* `idle' may be invoked without any args
* reformat some comments
* reformat usage string in program and man page
* call updateremotepwd() after successful login, not after successful connect
* always call setsockopt(, IPPROTO_IP, IP_TOS, ) (et al); using #if
defined(IPPROTO_IP) doesn't work on certain foreign systems where
enums instead of #defines are used...
[noted by Matthias Pfaller <leo@dachau.marco.de>]
 1.59 12-Nov-1999  lukem - implement `set rprompt'; right side version of `set prompt'. depends on
EL_RPROMPT support i added to editline(3).
- allow $FTPPROMPT and $FTPRPROMPT to override defaults for the relevant
prompts
- move `%' formatting code from prompt() to expandbuf().
- implement `%.' and `%c', similar to the same % codes in tcsh(1)
(functionality I added to tcsh nearly 6 years ago), except that `%.'
always does `...trailing' and `%c' always does `/<x>trailing'.
- unknown `%foo' codes get printed as `%foo'
 1.58 11-Nov-1999  lukem - implement updateremotepwd(); update the global variable `remotepwd'
to contain the remote working directory.
- add `set prompt', a user configurable prompt. (defaults to `ftp> ').
the following escape characters a la tcsh(1) are supported: %/, %m,
%M, and %n.
- add global var `username'; used by prompt code
- fix a couple of minor memory leaks
- bump version
 1.57 10-Nov-1999  lukem - implement restarting file:/// non-proxied http:// URLs (with -R).
- fix a semicolono which stopped file:/// from working
 1.56 24-Oct-1999  lukem new features:
- add `usage'; displays the usage of a command.
implemented by calling the c_handler() with argc = 0, argv = "funcname".
- add `passive auto'; does the same as $FTPMODE=auto.
- add `set [option value]'; display all options, or set an option to a value.
- add `unset option'; unset an option.
- add getoptionvalue() to retrieve an option's value, and replace a few
global variables with calls to this.
- implement cleanuppeer(), which resets various bits of state back to
`disconnected'. call in disconnect() and lostpeer().
- support completing on `options'.
- improve recovery after a SIGINT may have closed the connection.
XXX: there's still a couple to fix

other stuff:
- various consistency fixes in the man page.
- ensure that the command usage strings in the code and man page match reality.
- mput/mget: check that the connection still exists before each xfer.
- minor cosmetic changes in confirm().
- set code correctly in sizecmd() and modtime()
- don't need \n in err() strings.
- change lostpeer to take an argument (rather than casting (sig_t)lostpeer
in signal handlers)
- knf and whitespace police.
 1.55 12-Oct-1999  lukem s/ARPANET/Internet/ ...
 1.54 12-Oct-1999  lukem a few user interface and cosmetic tweaks:
* confirm(): move from util.c to cmds.c. display mnemonic string in its prompt.
add support for `q' (terminate current xfer), `?' (show help list)
* in various signal handlers, output a linefeed only if fromatty.
* if fgets(stdin) returned NULL (i.e, EOF), clearerr(stdin) because you don't
want future fgets to fail. this is not done for the fgets() in the main
command loop, since ftp will quit at that point.
* unless ftp is invoked with -a, don't retain the anonftp setting between
hosts (`ftp somehost:' sets anonftp, but you don't want that to `stick'
if you close that connection and open a new one).
 1.53 05-Oct-1999  lukem * factor out SIGINFO setting into a handler that is always active (but only
prints out info if bytes > 0). only set the handler if SIGINFO is defined
* hijack SIGQUIT to be the same as SIGINFO (foreign ports have this, and it's
annoying to have SIGQUIT dump core on netbsd when it prints info on other
systems)
* in {recv,send}request(), factor a lot of duplicated code out into a
`cleanup' section at the end
* rework shell() a bit
 1.52 05-Oct-1999  lukem add TNFi copyright to all files i've done more than a minor amount of work to...
 1.51 05-Oct-1999  lukem enhancments from Marc Horowitz <marc@mit.edu> to improve connection timeouts:
* implement xsignal_restart(), which only sets the SA_RESTART flag if
specifically requested
* xsignal() is now a wrapper to xsignal_restart(). INFO, USR1, USR2 and WINCH
are restartable, ALRM, INT, PIPE and QUIT are not.
* improve getreply()'s timeout code to take advantage of the above.

other changes:
* improve wording of how globbing works for `classic' URLs (host:path).
suggested by John Refling <johnr@imageworks.com> in relation to PRs
[bin/8519] and [bin/8520]
* always compile in the `edit' command even if NO_EDITCOMPLETE defined.
it's just a no-op in the latter case, which is more consistent to
the users.
* always compile in about: support (i.e, remove NO_ABOUT).
i'm entitled to some vanity in this program...
* clean up some whitespace
 1.50 01-Oct-1999  lukem If EPSV or EPRT fails, disable epsv4 for the rest of the current connection.
the disabled state can be overridden by toggling epsv4.

(I got sick of the errors about EPSV not being supported on almost
every server I connect to. This way we retain support for epsv4, but
it's not so whiny after the first failure...)
 1.49 01-Oct-1999  lukem * newer takes an optional second arg, so document this
* be a bit more explicit that the arguments to rate are in bytes
 1.48 28-Sep-1999  lukem * add new commands:
lpage page local files
pdir as dir, but through your $PAGER
pls as ls, but through your $PAGER
* implement docase() (a la dotrans() et al) and use appropriately, rototilling
some duplicated code
* globulize(): modify to return a pointer to the strdup()ed result in all cases,
and hack the code that calls it to take this into account
* replace strcpy() and strncpy() with strlcpy()
* put(), getit(): use some aptly named local vars instead of argv[...]
* delint
 1.47 22-Sep-1999  lukem branches: 1.47.2;
* add support for `xferbuf', which sets both `sndbuf' and `rcvbuf'
* document the above three commands
* rototill the way the sndbuf and rcvbuf work. remove resetsockbufsize()
* use the appropriate socket buffer size as the size of the buffer that
the read()/write() loops use. speeds up things in some cases.
 1.46 12-Jul-1999  lukem * change fetch_ftp() to be fully rfc 1738 compliant; if the URL contains
the an empty directory (e.g, between `some' and `path' in
`ftp://host/some//path'), then execute `CWD ' (without a path).
This command will probably fail on rfc 959 compliant servers, so
issue a warning in this case and bail. [noted by cgd].
(i wonder if the people who wrote rfc 1738 actually realised that this
requirement appears to contravene the spec for `cwd' in rfc 959 ?)
* replace isurl() with isipv6addr(), and use appropriately. fixes
auto-login with `classic ftp URLs' (e.g, `ftp somehost:')
* cleanup and rework some of the ipv6 stuff in parse_url()
* prevent potential coredump in fetch_ftp() when parsing `;type=X'
* KNF a few lines
* fix a couple of comments
* cleanup the man page a bit
 1.45 11-Jul-1999  itojun add epsv4 command, which enable/disable the use of EPSV/EPRT.

this is mainly for (hypothetical) ftp server which disconnect clients
that use EPSV/EPRT. I've never seen any ftp server like this, but
epsv4 command may be of use when such an ftp server is found.
 1.44 02-Jul-1999  itojun add dual-stack (IPv4/v6) support. hope I broke no other part...
 1.43 02-Jul-1999  itojun IPv6 support.
 1.42 29-Jun-1999  lukem [fear this; more ftp hacking from lukem :-]

features:
 1.41 13-Jun-1999  kleink Elaborate a bit on PAGER.
 1.40 02-Jun-1999  lukem * fix gate mode to login as `user@realhost' rather than using PASSERVE;
the latter only seemed to work for TIS Gauntlet and not TIS fwtk.
thanks to simonb@netbsd.org for testing this. fixes [bin/5556].
* if EOF (e.g, ^D) is entered at a username/password/account prompt which
happens to use fgets(), exit the login rather than treating EOF as CR.
* don't use the comma operator where separate statements are valid
* always use snprintf to copy stuff into malloced buffers, just in case
typos creep in and mean that the buffer ends up being overflowed
 1.39 12-May-1999  lukem parse http:// urls (and the $http_proxy variable) for [user:[pass]@]
elements, which are used for the initial authentication attempt (if
requested by the server). in the case of $http_proxy, use the values
for proxy authentication.
 1.38 04-May-1999  lukem clarify what uses gate-ftp mode. suggested by simonb
 1.37 28-Apr-1999  lukem * make parsing of ftp:// urls more RFC 1738 compliant;
- the path is split on `/', and each directory is CWD-ed into separately.
(from [standards/7484] by Alan Barrett <apb@iafrica.com>)
- support a trailing `;type=X' suffix, where X is a,i, or d. (d isn't
implemented, but it is recognised)
- the only non-compliant behaviour is that empty directories sections
(e.g `//') aren't run as `CWD ' - as a lot of ftpds don't like that.
Instead, treat this as a no-op.
* don't support globbing for ftp urls, since that's technically not
RFC compliant.
* fix a couple of man-page nits
 1.36 22-Mar-1999  lukem branches: 1.36.2;
* implement -R; restart non-proxied command-line FTP xfers
* fix fetch_ftp() so that hcode parsing is not done for file:// urls
(a } in the wrong place, and code at the wrong indent level...)
* change outfile to being a global (so it gets correctly reset)
* change parse_url to not remove leading '/' for non ftp urls.
whilst this is not totally rfc1738 compliant, other code kinda
assumes this is the case, and it doesn't hurt
 1.35 15-Mar-1999  garbled First round of .Os cleanups. .Os is defined in the tmac.doc-common file,
so we shouldn't override it with versions in the manpages. Many more to
come.
 1.34 08-Mar-1999  lukem * add url_decode() - `in-place' decode %xx escapes in a given url component
* parse_url()
- only look for user[:pass] for an ftp url (per rfc1738)
- strip leading /'s in an ftp url. (almost per rfc 1738)
* fetch_url()
- decode a copy of the path and use that to build local filenames
- send port in http Host: header (suggested by cgd@netbsd.org)
* fetch_ftp()
- url_decode() the user, pass and path
- fix splitting of path into dir & file (partially from [bin/7073])
- don't bother caching the last host; it can cause problems when
using ftp:// transfers, or when the user changes between xfers
* improve documentation of auto-fetched url arguments (especially regarding
escape sequences in ftp:// urls)
* some whitespace & copyright updates

this should fix [bin/7073] William O Ferry <woferry@warp.wofme.com>,
as well as the metaissues raised in that PR.
 1.33 07-Mar-1999  mycroft Clean up SYNOPSIS formatting.
 1.32 24-Jan-1999  lukem * -v enables verbose & progress, -V disables both
* set setvbuf(ttyout, NULL, _IOLBF, 0) and remove a bunch of fflush(ttyout).
* use fwrite() instead of write() for progressmeter (don't intermix stdio
with non stdio ops)
 1.31 23-Jan-1999  lukem support -f, which forces a cache flush for http xfers using either
`Pragma: no-cache' (http/1.0) or `Cache-Control: no-cache' (http/1.1)
 1.30 08-Oct-1998  lukem * hash & progress are mutually exclusive. [noted by mrg@netbsd.org]
* in autofetch mode, don't attempt to 'cd /' on first xfer (only on
subsequent xfers), as some ftp daemons don't permit that.
[noted by dbj@netbsd.org].
XXX: code could be smarter (but a lot more complicated to boot); this
works for now.
 1.29 08-Aug-1998  lukem default anon password is user@ not user@host
 1.28 03-Aug-1998  lukem features:
* support $no_proxy, which is a comma or space separated list of
host[:port] elements for which proxying is to be disabled.
(asked for by cgd in [bin/5027])
* if $FTPANONPASS is defined, use that as the anon ftp password
(instead of "`whoami`@")
* allow http URL's without a filename as long as an output file
is specified.

other stuff:
* implement parse_url(), which breaks up a URL into its bits, and use.
* simplify url_get() and auto_fetch() to use parse_url() and to not
modify the supplied URL or a copy of it.
* implement xmalloc() and xstrdup(); error-checked malloc()/strdup()
* add more consistency to messages, quoting strings in output as `%s'
 1.27 26-Jul-1998  lukem * spell `retrieved' correctly!
* parse Last-Modified: HTTP headers for date, and change time of
retrieved file if parsing succeeded
* support file:/// and file://localhost/ URLs
* in url_get(), re-write byte moving code to consistently use fread()
and fwrite(), and check errors at end with ferror()
* add about:* easter-egg (#ifndef SMALL :-)
 1.26 13-Jul-1998  perry Style police: "uses" is better than "utilizes", which is a horrible nelogism.
 1.25 04-Jun-1998  lukem some fixes & enhancements from openbsd's ftp, with extra fixes by me:
* default to passive with active fallback. $FTPMODE modifies this behaviour.
-A forces active connection.
* support '-o outfile' for auto-fetched files. outfile can be a file,
`-' (for stdout), or '|command' (to output each file through command).
* support '-r waittime', which retries the connection after waittime seconds
if it fails.
* fix 'page file' when restart is non-zero.
* try all ip-addresses of a host in a http fetch (as the normal ftp fetch
does).

XXX: a ``broken pipe'' error sometimes occurs with -o '|command';
i haven't tracked this down yet.
 1.24 20-May-1998  msaitoh fix typo.
 1.23 23-Aug-1997  lukem branches: 1.23.2;
support $TMPDIR
 1.22 18-Aug-1997  lukem bugs fixed:
* don't interpret '-' or '|' when a local filename is determined from
the remote name (i.e, in mget, and in get with only one argument).
This is implemented using an extra argument to recvrequest().
Fixes a major security hole.
* clean up memory leak when using globulize()
* clean up a couple of comments
* fix wording in TNF copyright

features added:
* support for TIS fwtk gate-ftp servers:
* read defaults from $FTPSERVER && $FTPSERVERPORT
* start in gate-ftp mode if invoked as 'gate-ftp'
* toggle or set with 'gate [host [port]]'
 1.21 10-Jun-1997  lukem * Only print the progress bar when ftp(1) is the foreground progress.
Patch from enami tsugutomo <enami@ba2.so-net.or.jp> in [bin/3735].
* Fix punctuation.
 1.20 14-Apr-1997  lukem More enhancements/bugfixes (when will it end?)
* differentiate between being connected, and being logged in
* cleanup some text messages
* support username & password ftp URLs (ftp://user:pass@host/) in non-proxy
situations; assume proxy supports it for proxy situations.
* cd to / before performing any autofetch transfers
* use strncasecmp in URL parsing. fix from <Todd.Miller@courtesan.com>
 1.19 05-Apr-1997  lukem * support $ftp_proxy for ftp:// transfers [bin/3245]
* add "more" & "less" as synonyms for "page"
* move editline setup code into controlediting(), and call appropriately.
only setup setup terminal if going into interactive mode. inspired
by Todd Miller <todd.miller@courtesan.com>
 1.18 13-Mar-1997  lukem Features:
* support remglobbing of auto_fetch arguments
* new flag - '-e'; disable editing
* "page file" == "get file |${PAGER-less}"

Bugfixes/cleanup:
* consistently use a trailing '.' on messages
* code cleanup, including buffer overrun fixes, use puts
and putchar in places, etc (inspired by OpenBSD mods)
* disable progress bar when local-file is a pipe or '-'
* skip \r in http headers
* fix remote ftpd slash bug more elegantly (so it works with ////)
* abort_remote(): check if cout==NULL before using it. should fix [bin/3273]
* fixed up cosmetic problems when complete_remote() generated errors from the
remote server (such as "no files found", "login with user and pass", ...)
done by adding extra argument to remglob(), which is a pointer to an error
buffer to put messages in rather than printing to stdout.
 1.17 01-Feb-1997  lukem [Yet Another Huge Ftp Commit - hopefully the last for a while,
barring any more little things people want added ...]

New features:
* progressmeter is now asynchronous, so "stalled" transfers can be
detected. "- stalled -" is displayed instead of the ETA in this case.
When the xfer resumes, the time that the xfer was stalled for is
factored out of the ETA. It is debatable whether this is better than
not factoring it out, but I like it this way (I.e, if it stalls for 8
seconds and the ETA was 30 seconds, when it resumes the ETA will still
be 30 seconds).
* verbosity can be disabled on the command line (-V), so that in auto-fetch
mode the only lines displayed will be a description of the file, and
the progress bar (if possible)
* if the screen is resized (and detected via the SIGWINCH signal), the
progress bar will rescale automatically.

Bugs fixed:
* progress bar will not use the last character on the line, as this can
cause problems on some terminals
* screen dimensions (via ioctl(TIOCWINSZ)) should use stdout not stdin
* progressmeter() used some vars before initialising them
* ^D will quit now. [fixes bin/3162]
* use hstrerror() to generate error message for host name lookup failure.
* use getcwd instead of getwd (it should have been OK, but why tempt fate?)
* auto-fetch transfers will always return a positive exit value upon failure
or interruption, relative to the file's position in argv[].
* remote completion of / will work, without putting a leading "///".
This is actually a bug in ftpd(1), where "NLST /" prefixes all names
with "//", but fixing every ftpd(1) is not an option...
 1.16 19-Jan-1997  lukem New features:
* Command line editing via editline(3) library.
* Context sensitive command and file completion, including remote files.

Enhancements to auto-fetch feature:
* Support for http:// URLs using the http protocol, including proxy HTTP
support via $htty_proxy if it's defined.
* The connection is kept open between successive files on the same host.
(obviously, this does not count for http requests.)
* Return value of ftp is 0 on no error, or the offset in argv[] of the file
which failed (i.e., argv[x] failed, ftp returns x).
* If the path in an ftp URL or classic format line has a trailing '/',
cd to the path and enter interactive mode. Fixes [bin/3011], albiet
requiring the user to help ftp in determining the operation.

Other changes:
* '-P port' works for normal ftp, and is the default for all classic style
auto-fetch transfers and for ftp URLs that don't specify the port.
(previously it would just work for the first xfer.)
* Some code moved into separated files along logical divisions.
* Editing and completion can be compiled out with -DSMALLFTP.
 1.15 06-Dec-1996  lukem * note that nlist and ls and equivalent
* fix up optional argument descriptions for dir, ls, nlist, and user.
 1.14 06-Dec-1996  lukem functionality mods:
* implement 'progress bar/meter' (inspired by ncftp). use 'progress' to
toggle on. it will display current file size to 5 digits, automatically
determining suffix (up to 16384 P (petabytes) == 2^64).
* 'ls' now uses NLST (unadorned listing), a la older ftp clients. 'dir'
still does LIST (long listing). idea from John Nemeth <jnemeth@cue.bc.ca>

bug fixes:
* return first line of reply in reply_string[] from getreply(), instead
of last line. This fixes [bin/741] (parsing of SYST), and also means
that SIZE and MDTM messages will be parsed correctly if they're longer
than 1 line.
* parse URL-style auto-ftps that have no filename correctly
(e.g, ftp://host, ftp://host/, ftp://host/dir/). pointed out by
Jaromir Dolecek <dolecek@saruman.ics.muni.cz>
* pass the correct size array in 2nd arg of utimes() when setting the
modification time
 1.13 28-Nov-1996  lukem More features, some of which were inspired by changes that
friedman@gnu.ai.mit.edu (Noah Friedman) made to his modified ftp:
- implement "lpwd" - local pwd
- implement "preserve" - toggle preserving of file modification
times on retrieved files
- allow for explicit "on" or "off" arg to toggle commands
- "exit" synonym for "quit", "msend" synonym for "mput"
- in confirmation mode, allow 'a' (yes to rest of current command),
and 'p' (turn off prompt mode, as if 'prompt off' was done,
effective immediately)
- "modtime" returns time formatted as localtime, not GMT

Bug fixes:
- check for extraneous args on commands
- cleanup const usage, line formatting
- create 0 length temporary file in remglob() to prevent symlink games
(from OpenBSD)
- check length of filename of ~/.netrc (from OpenBSD)
 1.12 25-Nov-1996  lukem cleanups/bugfixes:
- don't echo 'ACCT' parameter when debugging (a la 'PASS')
- Fix checking of directory access for "/foo", the parent
directory is "/", not "" (from FreeBSD)
- remove trailing whitespace on lines
- add any missing NetBSD tags
- cleanups to man page, includinging sorting options description

feature additions:
- variable sized hash marks (from [bin/683], but done in the hash command
as an optional arg)
- more user-friendly transfer time printing (from FreeBSD, with mods)
- '-p' command line option to jump into PASV mode (closes [bin/2857],
but with an option rather than checking argv[0])
- SIGINFO support for printing xfer stats when sending/receiving requests
- '-P port' for changing the port to connect to (from thorpej@netbsd.org)
- '-a': bypass normal login, and try anonymous login (from OpenBSD
via thorpej)
- autofetch files via url (ftp://...) or "classic" (host:/file)
(from OpenBSD via thorpej)
- 'ftp' synonymous with 'open' (from FreeBSD)
 1.11 08-Sep-1995  tls Sync with 4.4lite2
 1.10 13-Jan-1995  jtc Describe -t (enable packet tracing) command line flag (PR #713).
Alphabetize description of command line flags.

Packet tracing is an unimplemented feature, I've decided to keep it's
documentation since ever other UN*X vendor I checked still does. Not
a particularly useful reason.
 1.9 29-Aug-1994  mycroft branches: 1.9.2;
Add RCS ids.
 1.8 25-Aug-1994  cgd passive mode support, from David Carrel <carrel@cisco.com>, and cleaned
up for the new ftp sources by me.
 1.7 25-Aug-1994  cgd the previous local changes
 1.6 25-Aug-1994  cgd clean up import, no local changes.
 1.5 03-Feb-1994  jtc spelling mistakes
 1.4 11-Jan-1994  jtc Fix spelling errors.
 1.3 01-Aug-1993  mycroft Add RCS indentifiers.
 1.2 22-Apr-1993  mycroft Fix various bugs in man pages (from 386BSD patch 130).
 1.1 21-Mar-1993  cgd branches: 1.1.1; 1.1.2;
Initial revision
 1.1.2.1 25-Aug-1994  cgd import from 4.4-Lite
 1.1.1.2 05-Sep-1995  tls imported from 44lite2
 1.1.1.1 21-Mar-1993  cgd initial import of 386bsd-0.1 sources
 1.9.2.2 29-Aug-1994  mycroft Add RCS ids.
 1.9.2.1 29-Aug-1994  mycroft file ftp.1 was added on branch netbsd-1-0 on 1994-08-29 03:09:14 +0000
 1.23.2.1 10-Nov-1998  cgd pull up rev(s) 1.24-1.30 from trunk. (feyrer)
 1.36.2.2 23-Jun-1999  perry pullup 1.40->1.41 (kleink)
 1.36.2.1 22-Jun-1999  perry pullup 1.36->1.40 (lukem)
 1.47.2.1 27-Dec-1999  wrstuden Pull up to last week's -current.
 1.63.2.1 23-Jun-2000  minoura Sync w/ netbsd-1-5-base.
 1.82.2.1 09-Jul-2002  lukem pull up revisions 1.84-1.85 (requested by lukem in ticket #432):
1.85
Various improvements in wording and marking,
from Alan Barrett <apb@cequrux.com>, with a bit of tweaking from me.
1.84
highlight that for ftp:// auto-fetches, read access is required on
intermediate directories because ftp(1) obeys RFC1738. for [bin/15419]
 1.101.2.1 08-Oct-2004  jmc Pullup rev 1.102-1.104 (requested by dan in ticket #909)

Note potentially surprising file-saving behaviour in case of HTTP redirects.
 1.109.2.3 24-Jul-2005  tron Pull up revision 1.112 (requested by lukem in ticket #606):
In macdef, point to $ command (from jmc@openbsd), and split paragraph (it was getting a bit big).
 1.109.2.2 24-Jul-2005  tron Pull up revision 1.111 (requested by lukem in ticket #606):
Use more macros, and use them more consistently.
Fix a few typos.
 1.109.2.1 24-Jul-2005  tron Pull up revision 1.110 (requested by lukem in ticket #606):
Improve the description of the macros.
Patch from David H. Gutteridge in PR 24296.
 1.120.2.2 06-Aug-2007  lukem reorder location of "-s srcaddr" in synopsis
 1.120.2.1 06-Aug-2007  lukem file ftp.1 was added on branch matt-mips64 on 2007-08-06 04:42:14 +0000
 1.121.2.1 09-Jan-2008  matt sync with HEAD
 1.123.4.1 18-May-2008  yamt sync with head.
 1.124.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.127.6.1 13-May-2009  jym Sync with HEAD.

Third (and last) commit. See http://mail-index.netbsd.org/source-changes/2009/05/13/msg221222.html
 1.131.8.2 27-Aug-2016  bouyer Apply patch, requested by nonaka in ticket #1375:
src/usr.bin/ftp/cmds.c: patch
src/usr.bin/ftp/fetch.c: patch
src/usr.bin/ftp/ftp.1: patch
src/usr.bin/ftp/ftp.c: patch
src/usr.bin/ftp/ftp_var.h: patch
src/usr.bin/ftp/main.c: patch
src/usr.bin/ftp/ssl.c: patch
src/usr.bin/ftp/ssl.h: patch
src/usr.bin/ftp/version.h: patch
Update ftp(1) to version 20150912, adding https via proxy support.
 1.131.8.1 17-Dec-2013  bouyer Apply patch, requested by tron in ticket #997:
usr.bin/ftp/Makefile patch
usr.bin/ftp/cmds.c patch
usr.bin/ftp/cmdtab.c patch
usr.bin/ftp/extern.h patch
usr.bin/ftp/fetch.c patch
usr.bin/ftp/ftp.1 patch
usr.bin/ftp/ftp.c patch
usr.bin/ftp/ftp_var.h patch
usr.bin/ftp/main.c patch
usr.bin/ftp/progressbar.c patch
usr.bin/ftp/ssl.c patch
usr.bin/ftp/ssl.h patch
usr.bin/ftp/util.c patch
usr.bin/ftp/version.h patch

Add HTTPS support to ftp(1).
 1.131.6.2 23-Jan-2013  yamt sync with head
 1.131.6.1 17-Apr-2012  yamt sync with head
 1.133.2.1 25-Feb-2013  tls resync with head
 1.134.8.1 05-Nov-2015  riz Pull up following revision(s) (requested by wiz in ticket #981):
usr.bin/ftp/ftp.1: revision 1.135
usr.bin/ftp/ssl.c: revision 1.3
usr.bin/ftp/ssl.c: revision 1.4
usr.bin/ftp/ssl.h: revision 1.3
usr.bin/ftp/version.h: revision 1.86
usr.bin/ftp/version.h: revision 1.87
usr.bin/ftp/fetch.c: revision 1.207
usr.bin/ftp/main.c: revision 1.123
Add -x xferbufsize to set xferbuf size.
Implement -x xferbufsize set the socket send and receive buffer size,
as per 'xferbuf' in interactive mode.
Patch from Nicholas Mills (via private mail), with minor adjustment by me.
Add Server Name Indication (SNI) support for https.
Needed for e.g. some github URLs.
Bump version for SNI support.
servername cannot be NULL here.
Noted by joerg@.
 1.135.8.3 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1763:

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/cmds.c up to 1.141
usr.bin/ftp/complete.c up to 1.47
usr.bin/ftp/domacro.c up to 1.23
usr.bin/ftp/extern.h up to 1.82
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp.c up to 1.174
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/progressbar.c up to 1.24
usr.bin/ftp/progressbar.h up to 1.9
usr.bin/ftp/ssl.c up to 1.12
usr.bin/ftp/ssl.h up to 1.5
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

ftp(1): validate address from PASV and LPSV response.
ftp(1): use raw write(2) instead of fwrite(3) to avoid stream
corruption because of the progress bar interrupts.
Fixes for PR 56219 and PR 55857.
PR 57003: Support relative redirects.
 1.135.8.2 12-Sep-2022  martin Backout ticket #1763 for now - trust anchors are not solved.
 1.135.8.1 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1763:

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/cmds.c up to 1.141
usr.bin/ftp/complete.c up to 1.47
usr.bin/ftp/domacro.c up to 1.23
usr.bin/ftp/extern.h up to 1.82
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp.c up to 1.174
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/progressbar.c up to 1.24
usr.bin/ftp/progressbar.h up to 1.9
usr.bin/ftp/ssl.c up to 1.11
usr.bin/ftp/ssl.h up to 1.5
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

ftp(1): validate address from PASV and LPSV response.
ftp(1): use raw write(2) instead of fwrite(3) to avoid stream
corruption because of the progress bar interrupts.
Fixes for PR 56219 and PR 55857.
PR 57003: Support relative redirects.
 1.136.8.4 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1523

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/ssl.c up to 1.12
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

PR 57003: Support relative redirects.
 1.136.8.3 12-Sep-2022  martin Back out ticket #1523 for now - trust anchor validation is not solved.
 1.136.8.2 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1523

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/ssl.c up to 1.11
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

PR 57003: Support relative redirects.
 1.136.8.1 14-Jun-2021  martin Pull up following revision(s) (requested by lukem in ticket #1294):

usr.bin/ftp/cmds.c: revision 1.141
usr.bin/ftp/ftp.1: revision 1.143

ftp(1): fix description of "debug"

"debug" command and documentation got accidentally renamed
to "ftp_debug" 13 years ago, and was only partially fixed.
 1.147.2.7 02-Dec-2024  martin Pull up following revision(s) (requested by lukem in ticket #1021):

usr.bin/ftp/ftp.c: revision 1.178
usr.bin/ftp/version.h: revision 1.99
usr.bin/ftp/ruserpass.c: revision 1.35
usr.bin/ftp/main.c: revision 1.134
usr.bin/ftp/main.c: revision 1.135
usr.bin/ftp/progressbar.c: revision 1.27
usr.bin/ftp/util.c: revision 1.165
usr.bin/ftp/cmds.c: revision 1.144
usr.bin/ftp/extern.h: revision 1.84
usr.bin/ftp/fetch.c: revision 1.242
usr.bin/ftp/ftp.1: revision 1.160

s/bninary/binary/ in comment.

extract duplicate code into a function.

Check bounds when copying to destination.

Remove const where the const string ended up being overwritten.

use unsigned when doing shifts.

remove const when string gets overwritten.

ftp: exit non-zero if short http transfer when filesize is known
If a http file size is known and the fetch finishes with less bytes
transferred, exit non-zero.
Bump version to 20241129.
PR bin/54713
PR bin/58281

ftp: help improvements
Document -? as a separate mode.
Document -H HEADER in the usage.
Clarify units for -b and -x.
Consistent argument names between ftp -? and ftp(1).

ftp: order getopt Upper before lower
Consistently order options in getopt and the switch
with the upper case option before the lower case option.
This makes it easier to cross-reference with -? and ftp(1).
No functional change.
 1.147.2.6 02-Dec-2024  martin Apply patch, requested by lukem in ticket #1020:

usr.bin/ftp/ftp.1 1.151 (apply patch)

Fix mis-application of this change in pullup #970
 1.147.2.5 13-Oct-2024  martin Pull up following revision(s) (requested by riastradh in ticket #970):

tests/usr.bin/Makefile: revision 1.42
usr.bin/ftp/ruserpass.c: revision 1.34
usr.bin/ftp/main.c: revision 1.130
usr.bin/ftp/ssl.c: revision 1.17
usr.bin/ftp/main.c: revision 1.131
usr.bin/ftp/ssl.c: revision 1.18
usr.bin/ftp/main.c: revision 1.132
usr.bin/ftp/ssl.c: revision 1.19
usr.bin/ftp/main.c: revision 1.133
distrib/sets/lists/tests/mi: revision 1.1342
usr.bin/ftp/ftp.1: revision 1.151
usr.bin/ftp/ftp.1: revision 1.152
usr.bin/ftp/progressbar.c: revision 1.25
usr.bin/ftp/ftp.1: revision 1.153
usr.bin/ftp/progressbar.c: revision 1.26
usr.bin/ftp/ftp.1: revision 1.155
usr.bin/ftp/ftp.1: revision 1.156
usr.bin/ftp/fetch.c: revision 1.239
usr.bin/ftp/ftp.1: revision 1.157
usr.bin/ftp/ftp.1: revision 1.158
usr.bin/ftp/ftp.1: revision 1.159
usr.bin/ftp/ftp_var.h: revision 1.87
etc/mtree/NetBSD.dist.tests: revision 1.208
usr.bin/ftp/ftp_var.h: revision 1.88
usr.bin/ftp/ftp_var.h: revision 1.89
usr.bin/ftp/cmds.c: revision 1.142
usr.bin/ftp/util.c: revision 1.168
usr.bin/ftp/cmds.c: revision 1.143
tests/usr.bin/ftp/custom_headers.sh: revision 1.1
usr.bin/ftp/ssl.c: revision 1.20
usr.bin/ftp/complete.c: revision 1.48
tests/usr.bin/ftp/Makefile: revision 1.1
tests/usr.bin/ftp/t_custom_headers.sh: revision 1.1
usr.bin/ftp/fetch.c: revision 1.240
usr.bin/ftp/fetch.c: revision 1.241
usr.bin/ftp/ftp.c: revision 1.176
usr.bin/ftp/ftp.c: revision 1.177
(all via patch)

ftp(1): wording and formatting improvements

Fix grammar issue with "Support values" reported in private mail.
Document all file transfer types in "type" and cross-reference that.
Consistency fixes in describing file transfer parameters and types.

Fix some mandoc -Tlint issues (except "useless macro: Tn").

Add -b <buflen> to specify the buffer size.

ftp: bump FTPBUFLEN from 4kB to 16kB
sourceforge.net returns a 5kB content-security-policy.
Analyzed by mlelstv@ who reports usual limits are between 4kB and 48kB.
default is now 16K

ftp: improve -b documentation

Order -b bufsize in the synopsis.

Document the actual default value.

ftp: improve units used in comments and errors
Use "KiB" instead of "K" in errors.
Clarify related comments.

pass some lint.

PR/58581: Sunil Nimmagadda: Add flag to allow specifying extra http header
fields.

ftp(1): Nix trailing whitespace in man page.
No functional change intended.

PR bin/58581: ftp(1) should allow specifying header fields in http requests
fix markup (h -> H), explain about multiple headers, fix usage (from RVP)

Don't forget the dot, use the intended macro name (I think),
and improve the wording a little. (All related to the -H option.)
ftp(1): Add test for custom HTTP header fields.

Based on a patch from Sunil Nimmagadda.

PR bin/58581: ftp(1) should allow specifying header fields in http
requests
 1.147.2.4 20-Jun-2024  martin Pull up following revision(s) (requested by gutteridge in ticket #704):

usr.bin/ftp/ftp.1: revision 1.154

ftp.1: drop a sentence that's no longer accurate
 1.147.2.3 16-May-2023  martin Pull up following revision(s) (requested by lukem in ticket #173):

usr.bin/ftp/ftp.1: revision 1.150

ftp(1): better mark up for url vs cd example
 1.147.2.2 16-May-2023  martin Pull up following revision(s) (requested by lukem in ticket #172):

usr.bin/ftp/ftp.1: revision 1.149

ftp(1): minor markup tweaks

Use .Ql instead of .Sq Li, add some missing ones. Use .Pq instead of
explicit () for longer phrases - these are easier to read in the
postscript output b/c of extra spacing.
 1.147.2.1 16-May-2023  martin Pull up following revision(s) (requested by lukem in ticket #170):

usr.bin/ftp/ssl.c: revision 1.13
usr.bin/ftp/fetch.c: revision 1.236
usr.bin/ftp/util.c: revision 1.166
usr.bin/ftp/main.c: revision 1.129
usr.bin/ftp/extern.h: revision 1.83
usr.bin/ftp/ftp.1: revision 1.148
usr.bin/ftp/cmdtab.c: revision 1.53
usr.bin/ftp/version.h: revision 1.96

Add option sslnoverify to control validation of SSL certificates.

Add netrc processing to fetch-mode (URL on command line) to enable options and autologin
via netrc.

Fix SSL cleanup in some error paths.

Certificate validation is now enabled by default. Set FTPSSLNOVERIFY=1 in environment
or configure a corresponding init macro via netrc to not validate certs (required if
you haven't installed a required CA certificate for OpenSSL).

Discussed with lukem@ on icb.
 1.154.2.1 02-Aug-2025  perseant Sync with HEAD
 1.178 04-Oct-2024  christos remove const when string gets overwritten.
 1.177 25-Sep-2024  christos pass some lint.
 1.176 18-Feb-2024  christos branches: 1.176.2;
Add -b <buflen> to specify the buffer size.
 1.175 05-May-2023  lukem add timeout for ssl connect

Implement a timeout for SSL connection setup, using -q QUITTIME,
defaulting to 60 seconds.
SSL_connect(3) (unlike connect(2)) doesn't timeout by default.

Adapt ssl error messages destination: if unexpected error
from local API, use warn()/warnx() to stderr;
if expected error from a network operation (e.g., timeouts),
use fprintf to ttyout (which might be stdout).

Consistently use ftp_poll() instead of select();
ssl.c (using select()) was added 7 years after the
previous uses of select() were converted to poll().

Check EAGAIN as well as existing EINTR error from ftp_poll(),
for portability.
 1.174 26-Aug-2021  lukem branches: 1.174.2;
ftp: remove unnecessary variable assignments

Remove assignment to error in initconn(); it's not tested anywhere after the
initial use, so no need to set it before goto bad.

(Looks like copypasta from the initial addition of the code in rev 1.48.)
 1.173 26-Aug-2021  lukem ftp: validate address from PASV and LPSV response

Fail if the server's response to PASV or LPSV contains an IP address
that doesn't match that of the control connection.
(EPSV already only uses the port portion of the server's response,
per RFC 2428).

Previously a hostile server could cause ftp to open a data connection elsewhere.

Many other ftp implementations have had a similar change for many years,
including those in popular browsers (before they deprecated FTP ...)

Thanks to Simon Josefsson notifying me about
https://lists.gnu.org/archive/html/bug-inetutils/2021-06/msg00002.html
 1.172 03-Jun-2021  lukem set SO_KEEPALIVE on control connection

Attempt to prevent timeouts of the control connection by setting SO_KEEPALIVE.
This matches the equivalent behaviour in ftpd.

Note: This is a much simpler change than adding a background polling event
to invoke "STAT" (or "NOOP") on the control connection during a transfer.
(It's unclear from RFC 959 whether "NOOP" is even permitted during a transfer).

PR bin/56129
 1.171 06-Jan-2021  lukem branches: 1.171.4;
ftp: don't use restartable signals

Refactor to not rely upon restartable signals (SA_RESTART),
possibly fixing intermittent failures with -q QUITTIME.

ftp transfers: handle EINTR/EAGAIN in copy_bytes(),
instead of relying upon restartable signals.

http/https transfers: Explicitly print an error similar to
progressmeter() when timing-out for -Q QUITTIME in fetch_wait(),
and set errno to ETIMEDOUT so that the warn() in fetch_url()
prints a more accurate error message.

PR/55857
 1.170 11-Jul-2020  lukem ftp.c: improve signal handler restoration

Only invoke the old signal handler if it's a real signal handler
and not SIG_IGN, SIG_DFL, SIG_HOLD, or SIG_ERR, using new static
function issighandler().
Avoids an intermittent race condition with a null pointer
dereference via (*SIG_DFL)().
Bug class reported by Joyu Liao from Juniper Networks.

Use SIG_ERR instead of NULL as the indicator that a signal handler
hasn't been changed, so that SIG_DFL (equivalent to NULL)
will be restored.
 1.169 08-Jun-2020  lukem ftp: exit if lostpeer invoked by a signal

lostpeer() calls too many async-unsafe functions (both directly
and indirectly) to close and cleanup the remote connections,
so just exit after the cleanup if invoked by a signal.

Reported in private mail by Qi Hou.
May also resolve a crash reported by Thomas Klausner.
 1.168 04-Feb-2019  mrg branches: 1.168.2;
- add justquit() that always exits. use it to avoid unreachable code.
 1.167 04-Oct-2016  joerg branches: 1.167.6; 1.167.14;
When using data outside the signed char range, it is better to
consistently use an unsigned char buffer.
 1.166 13-Dec-2015  tron branches: 1.166.2;
(Hopefully) fix build without IPv6 support
 1.165 11-Dec-2015  tron Use the proper format "[IPv6 address]:port" when reporting connection
attempts to IPv6 endpoints.
 1.164 04-Jul-2012  is branches: 1.164.10;
As discussed on tech-net@: Don't display expected EHOSTUNREACH for all but
the last connect attempts in terse mode.
 1.163 10-Dec-2011  lukem branches: 1.163.2;
Move determination of socket buffer sizes from startup to the first
time a socket is used, as the previous logic assumed AF_INET sockets
were available (which they may not be in an IPv6-only system).
Per discussion with Maxim Konovalov and the FreeBSD problem 162661.
 1.162 16-Sep-2011  joerg branches: 1.162.2;
Use __dead
 1.161 14-Aug-2011  christos fix gcc-4.5 warnings
 1.160 05-Mar-2010  lukem Back to using 'RFC xxxx' instead of 'RFCxxxx'
 1.159 15-Apr-2009  jld Unbreak the build by adding curly braces to placate the empty-body warning.
 1.158 12-Apr-2009  lukem Fix numerous WARNS=4 issues (-Wcast-qual -Wsign-compare).
 1.157 12-Apr-2009  lukem fix -Wshadow issues
 1.156 10-May-2008  skd branches: 1.156.6;
Add epsv6 and epsv to disable extended passive mode for ipv6 or both ipv4 and ipv6 respectively. This hack is due to our friends a Juniper Networks who break
epsv in ipv6. Should be fixed in ScreenOS 6.2.X.
 1.155 28-Apr-2008  martin branches: 1.155.2;
Remove clause 3 and 4 from TNF licenses
 1.154 22-Apr-2008  lukem Use the service name to getaddrinfo() (along with the host name), so that
features such as DNS Service Discovery have a better chance of working.
Suggested by David Young <dyoung>.

Display the service name in various status & error messages.

Don't getservbyname() the :port component of a URL; RFC 3986 says it's
just an unsigned number, not a service name.
 1.153 05-Dec-2007  lukem branches: 1.153.6;
Rename HAVE_STRUCT_SOCKADDR_SA_LEN to HAVE_STRUCT_SOCKADDR_IN_SIN_LEN
to accurately reflect the structure member being used.
 1.152 22-Jul-2007  lukem branches: 1.152.4; 1.152.6;
Replace HAVE_SOCKADDR_SA_LEN with defined(HAVE_STRUCT_SOCKADDR_SA_LEN)
 1.151 24-May-2007  lukem Display times in RFC2822 form rather than using ctime(3), since
the former is more explicit about the timezone offset.
 1.150 15-May-2007  lukem * Modify parse_url() to consistently strip the leading `/' off ftp URLs.
Fixes PR 17617.
* Use 'RFCnnnn' (with leading 0) instead of 'RFC nnnn', to be
consistent with the style in the RFC index.
* Refer to RFC3916 instead of 1738 or 2732.
* Expand the list of supported RFCs in ftp(1) to contain the document
name as well.
 1.149 10-May-2007  lukem Implement copy_bytes() to copy bytes from one fd to another via the
provided buffer, with optional rate-limiting and hash-mark printing,
using one loop and handle short writes.
Refactor sendrequest() and recvrequest() to use copy_data().
Addresses PR 15943.
 1.148 18-Apr-2007  lukem fix rev 1.144: initconn() may be called with verbose==-1 (e.g., during remote
completion), so only print out the successful EPSV response if verbose>0.
 1.147 17-Apr-2007  lukem * Implement -s srcaddr; uses srcaddr as the local IP address for all
connections.
Based on code in the version of ftp that FreeBSD had before they
replaced it with lukemftp.
* Move error message handling into ftp_connect() rather than in the
caller, so that more specific error reporting can occur.
* Improve consistency of various warning and error messages.
 1.146 16-Apr-2007  lukem Replace a "while" with an "if" since the code path only gets executed once.
 1.145 12-Apr-2007  lukem whitespace pedantry
 1.144 11-Apr-2007  lukem Suppress printing non-COMPLETE reply strings from EPSV and EPRT, as we're
going to fall back to PASV / PORT (respectively) if the former fail,
and this avoids printing a failure reply followed by a success reply.
Should fix a problem with the emacs ftp wrapper.
 1.143 13-Dec-2006  christos gcc4 does not care about &foo; use volatile instead. From Anon Ymous
 1.142 23-Oct-2006  christos simplify 421 printing code (jani at xeebioneurope dot de)
 1.141 07-Oct-2006  elad PR/32855: der Mouse: [dM] ftp -q is broken (or misdocumented)

Patch applied, thanks!
 1.140 10-May-2006  mrg quell GCC 4.1 uninitialised variable warnings.

XXX: we should audit the tree for which old ones are no longer needed
after getting the older compilers out of the tree..
 1.139 28-Apr-2006  christos Coverity CID 874: Don't check local; it cannot be NULL.
 1.138 28-Apr-2006  christos Coverity CID 875: local is not allowed to be NULL; don't check for it.
 1.137 31-Jan-2006  christos rename debug to ftp_debug. grr libssh.
 1.136 31-Jan-2006  christos Rename xfoo() to ftp_foo() to avoid collisions with libssh. Don't ask.
 1.135 29-Jun-2005  christos Add NO_USAGE and NO_DEBUG so that we can fit in the floppies again.
 1.134 10-Jun-2005  lukem Implement:
int getline(FILE *stream, char *buf, size_t buflen, const char **errormsg)
Read a line from the FILE stream into buf/buflen using fgets(), so up
to buflen-1 chars will be read and the result will be NUL terminated.
If the line has a trailing newline it will be removed.
If the line is too long, excess characters will be read until
newline/EOF/error.
Various -ve return values indicate different errors, and errormsg
will be changed to an error description if it's not NULL.

Convert to use getline() instead of fgets() whenever reading user input
to ensure that an overly long input line doesn't leave excess characters
for the next input operation to accidentally use as input.

Zero out the password & account after we've finished with it.

Consistently use getpass(3) (i.e, character echo suppressed) when
reading the account data. For some reason, historically the "login"
code suppressed echo for Account: yet the "user" command did not!

Display the hostname in the "getaddrinfo failed" warning.

Appease some -Wcast-qual warnings. Fixing all of these requires
significant code refactoring. (mmm, legacy code).
 1.133 01-Jun-2005  lukem * Only print the "Trying <address>..." message if verbose and
there's more than one struct addrinfo in the getaddrinfo() result.
* Don't use non-standard "u_int".
 1.132 14-May-2005  lukem Fix some cast issues highlighted by Scott Reynolds using gcc 4 on OSX.4
 1.131 13-May-2005  lukem * Correct the "optlen" argument passed to getsockopt(3) and setsockopt(3)
in various places. Fixes a problem noted by Allen Briggs.
* Improve warning printed when connect(2) for the data channel fails.
 1.130 11-May-2005  lukem Use socklen_t instead of int as the 5th argument to getsockopt().
Improve invocation of setsockopt() and associated failure messages.
 1.129 11-Apr-2005  lukem typo in previous
 1.128 11-Apr-2005  lukem gratuitous whitespace cleanup (before someone else jumps the gun...)
 1.127 11-Apr-2005  lukem Implement a timeout on the accept(2) in dataconn() and the
connect(2) in xconnect() by temporarily setting O_NONBLOCK
on the socket and using xpoll() to wait for the operation
to succeed.
The timeout used is the '-q quittime' argument (defaults to
60s for accept(2), and the system default for connect(2)).
Idea inspired by discussion with Chuck Cranor.
This may (indirectly) fix various problems with timeouts
in active mode through broken firewalls.

Implement xpoll() as a wrapper around poll(2), to make it
easier to replace on systems without a functional poll(2).
Unconditionally use xpoll() instead of conditionally using
select(2) or poll(2).
 1.126 20-Jul-2004  lukem branches: 1.126.2;
If an ftp auto-fetch transfer is interrupted by SIGINT (usually ^C),
exit with 130 instead of 1 (or rarely, 0).
This allows an ftp auto-fetch in a shell loop to correctly terminate the loop.
Should fix PR [pkg/26351], and possibly others.
 1.125 10-Apr-2004  lukem If connect(2) in xconnect() fails with EINTR, call select(2) on the socket
until it's writable or it fails with something other than EINTR.
This matches the behaviour in SUSv3, and prevents the problem when
pressing ^T (SIGINFO, which is marked as restartable) during connection
setup would cause ftp to fail with EADDRINUSE or EALREADY when the
second connect(2) was attempted on the same socket.
Problem found and solution provided by Maxime Henrion <mux@freebsd.org>.
 1.124 10-Apr-2004  lukem whitespace consistency tweak
 1.123 10-Dec-2003  lukem Don't warn about "ignored setsockopt" failures unless debugging is
enabled. Suggested by Todd Vierling.

Allow empty passwords in ftp://user:@host/file auto-fetch URLs,
per RFC 1738. Requested by Simon Poole.

Update version.
 1.122 07-Aug-2003  agc Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22365, verified by myself.
 1.121 31-Jul-2003  lukem Invalidate remote directory completion cache if any command which
may change the remote contents completes successfully, including:
del, mdel, ren, mkdir, rmdir, quote, and all upload commands
Patch from Yar Tikhiy <yar@comp.chem.msu.su>.
 1.120 05-Jun-2002  lukem - when showing the final progress bar, replace "00:00 ETA" with the
elapsed time. (suggested by simonb)
- actually display transfer stats after a URL fetch. (bug introduced a
*long* time ago)
- update copyright & version
 1.119 07-May-2002  lukem Use "r+" instead of "r+w", since the latter is not standard.
Noted by <Steve.McClellan@radisys.com> in private email.
 1.118 25-Apr-2002  itojun avoid buffer overrun on PASV from malicious server.
http://online.securityfocus.com/archive/1/269356/2002-04-22/2002-04-28/0
 1.117 26-Dec-2001  lukem update copyrights
 1.116 23-Dec-2001  lukem Add -4 to force IPv4 and -6 to force IPv6 address usage.
From Hajimu UMEMOTO, via Mike Heffner of FreeBSD.

(FreeBSD has imported NetBSD's ftp as their ftp client;
Mike is sending back some of their local changes).
 1.115 20-Dec-2001  lukem Large file ASCII mode support by using fseeko() instead of fseek().
From Andrey A. Chernov of FreeBSD, via Mike Heffner.
 1.114 19-Feb-2001  lukem minor knf
 1.113 27-Nov-2000  itojun fix INET6-less build (like x_ftp). PR11578
 1.112 24-Nov-2000  itojun cope with 2553bis getnameinfo (always attach scope id)
getnameinfo error check.
 1.111 15-Nov-2000  itojun use NI_MAXHOST with getnameinfo. we can assume presense of getnameinfo.
 1.110 11-Oct-2000  is More format string cleanup by sommerfeld.
 1.109 28-Sep-2000  lukem explicitly use SOCK_STREAM with socket() instead of res->ai_socktype,
because it appears that linux with glibc doesn't set the latter
correctly after one of getaddrinfo() or getnameinfo().
 1.108 06-Aug-2000  lukem * implement parseport(), which takes a string and attempts to convert
it to a numeric port number
* use parseport() in parse_url() and hookup()
* don't try and lookup the port number using getaddrinfo(), as it's too hard
to separate a failed host name lookup from a failed service name lookup.
this was causing lossage on systems that don't have `http' in services(5)
(such as solaris), but only crept in when we started using getaddrinfo()
unconditionally.
 1.107 01-Aug-2000  lukem - rename NO_QUAD to NO_LONG_LONG, QUAD* -> LL* and add ULL* (unsigned)
equivalents. name change suggested by Klaus Klein <kjk@netbsd.org>
- change defined(BSD4_4) || HAVE_SIN_LEN tests into HAVE_SOCKADDR_SA_LEN,
and set the latter if BSD4_4 exists
 1.106 31-Jul-2000  lukem - we can't just rename BSD4_4 -> HAVE_SIN_LEN, since bsd systems define BSD4_4;
change tests to test for either defined(BSD4_4) or HAVE_SIN_LEN
- more KNF
 1.105 30-Jul-2000  lukem * always set (struct sockinet).su_len after getsockname() et al, so
that it's valid on systems which don't have sin_len and need the `compat'
version
* fix the accept() in dataconn() to use the correct struct elem
 1.104 30-Jul-2000  lukem * always use getaddrinfo() and getnameinfo() instead of maintaining two code
paths. (lukemftp will provide replacements for these on older systems)
* rename __USE_SELECT to USE_SELECT
* rename BSD4_4 to HAVE_SIN_LEN
* replace union sockunion {} with struct sockinet {}, and modify the code
accordingly. this is possibly more portable, as it doesn't rely upon the
structure alignment within the union for our own stuff.
(XXX: haven't tested the ipv6 stuff)
 1.103 30-Jul-2000  lukem clean up NO_QUAD support: create helper #defines and use as appropriate:
#define NOQUAD ! NOQUAD
------- ------ - ------
QUADF "%ld" "%lld"
QUADFP(x) "%" x "ld" "%" x "lld"
QUADT long long long
STRTOL(x,y,z) strtol(x,y,z) strtoll(x,y,z)
 1.102 18-Jul-2000  lukem add support for FEAT and OPTS commands with `features' and `opts'.
(from RFC 2389).

add support for MLST & MLSD (machine parseble listings) with 'mlst', 'mlsd'
and 'pmlsd' (mlsd |$PAGER) commands. (from draft-ietf-ftpext-mlst-11)

rename remotesyst() to getremoteinfo(), and modify to parse the result from
FEAT (if supported), and take into account the support for the various
extensions such as MDTM, SIZE, REST (STREAM), MLSD, and FEAT/OPTS.
put each feature into one of the following categories:
- known to work (explicit FEAT)
- unknown but assume works until explicit failure, when it's
then tagged as `known not to work'.
- known not to work (FEAT succeeded but didn't return anything,
or was unknown and then explicit failure)
assign results into features[] matrix.

add support to getreply() so that an optional callback will be called
for each line received from the server except for the first and last.
this is used in FEAT (and MLST) parsing.

modify various commands to check if REST (STREAM), MDTM and SIZE are
explicitly or implicitly supported before using.

fix `syst' when verbose is off.

minor knf (indent goto labels by one space, etc).

simply various command usage handlers by assuming that argv != NULL except
for quit() and disconnect().
 1.101 07-Jul-2000  itojun errx?/warnx? audit. do not pass variable alone, use %s. idea from openbsd
 1.100 11-Jun-2000  lukem branches: 1.100.2;
from itojun: better fix for previous (doesn't need in_addr_t or u_int32_t)
 1.99 11-Jun-2000  lukem portability fixes for lukemftp:
* initconn(): use in_addr_t instead of u_int32_t when manipulating IPv6
addresses (and assume anything with ipv6 has in_addr_t; if not, i'll
add an autoconf test for it)
* ai_unmapped(): not all systems have sin_len; so only set #ifdef BSD4_4
* fix some lint
 1.98 05-Jun-2000  lukem - fix ai_unmapped() to be a no-op in the !def INET6 case
- display `(-INET6)' at the end of the version string if !def INET6
- clarify in the man page that IPv6 support may not be present (for lukemftp :)
 1.97 30-May-2000  itojun more comment on IPv4 mapped address handling.
 1.96 29-May-2000  itojun convert IPv4 mapped address (::ffff:10.1.1.1) into real IPv4 address
before touching it. IPv4 mapped address complicates too many things
in FTP protocol handling.
 1.95 01-May-2000  lukem branches: 1.95.2;
convert to ANSI KNF
 1.94 01-May-2000  lukem * Add support for `fget localfile', which reads a list of filenames to
retrieve from localfile. Based on work by Darren Reed.
* Crank version.
* Update copyright dates.
 1.93 14-Mar-2000  itojun inhibit too-noisy message for scoped address data transfer
(will be enabled in "debug" mode).
 1.92 28-Feb-2000  lukem only use IPTOS_ setsockopt()s if they're defined (e.g, SunOS doesn't).
from Havard.Eidnes@runit.sintef.no
 1.91 14-Feb-2000  lukem only use getaddrinfo() et al if both NI_NUMERICHOST *and* INET6 are defined...
(allows --disable-ipv6 in lukemftp's configure script to disable this as
well, which is good for testing when it appears getaddrinfo() is borken)
 1.90 31-Jan-2000  lukem define private type `sigfunc' as
typedef void (*sigfunc) __P((int));
and replace use of sig_t and void (*)(int).

certain other OSes define sig_t differently to that (they add extra arguments),
and it causes problems due to function mismatches, etc...
 1.89 11-Dec-1999  lukem separate out the main `data pump' loop into two: one that supports
rate limiting and one that doesn't. simplifies the code, and speeds
up the latter case a bit, at the expense of duplicating a few lines...
 1.88 26-Nov-1999  lukem * complete_remote(): use remglob("", ...) instead of remglob(".", ...),
for listings of the current working directory; some ftp servers don't
like `NLST .'.
[noted by Giles Lean <giles@nemeton.com.au>]
* recvrequest(): treat remote=="" as remote==NULL when calling command().
(to support the above change)
* support `[user@]' in `[user@]host' and `[user@]host[:][path]'.
[based on idea (and initial code) from David Maxwell <david@fundy.ca>]
* `idle' may be invoked without any args
* reformat some comments
* reformat usage string in program and man page
* call updateremotepwd() after successful login, not after successful connect
* always call setsockopt(, IPPROTO_IP, IP_TOS, ) (et al); using #if
defined(IPPROTO_IP) doesn't work on certain foreign systems where
enums instead of #defines are used...
[noted by Matthias Pfaller <leo@dachau.marco.de>]
 1.87 11-Nov-1999  lukem whitespace nits
 1.86 03-Nov-1999  lukem hookup(): when using getservbyname() (when getaddrinfo() isn't available), if
the provided port is a valid number use that rather than trying to do
getservbyname() against it.
fixes a problem on foreign systems noted by Chuck Silvers <chuq@chuq.com>
 1.85 24-Oct-1999  lukem new features:
- add `usage'; displays the usage of a command.
implemented by calling the c_handler() with argc = 0, argv = "funcname".
- add `passive auto'; does the same as $FTPMODE=auto.
- add `set [option value]'; display all options, or set an option to a value.
- add `unset option'; unset an option.
- add getoptionvalue() to retrieve an option's value, and replace a few
global variables with calls to this.
- implement cleanuppeer(), which resets various bits of state back to
`disconnected'. call in disconnect() and lostpeer().
- support completing on `options'.
- improve recovery after a SIGINT may have closed the connection.
XXX: there's still a couple to fix

other stuff:
- various consistency fixes in the man page.
- ensure that the command usage strings in the code and man page match reality.
- mput/mget: check that the connection still exists before each xfer.
- minor cosmetic changes in confirm().
- set code correctly in sizecmd() and modtime()
- don't need \n in err() strings.
- change lostpeer to take an argument (rather than casting (sig_t)lostpeer
in signal handlers)
- knf and whitespace police.
 1.84 12-Oct-1999  lukem a few user interface and cosmetic tweaks:
* confirm(): move from util.c to cmds.c. display mnemonic string in its prompt.
add support for `q' (terminate current xfer), `?' (show help list)
* in various signal handlers, output a linefeed only if fromatty.
* if fgets(stdin) returned NULL (i.e, EOF), clearerr(stdin) because you don't
want future fgets to fail. this is not done for the fgets() in the main
command loop, since ftp will quit at that point.
* unless ftp is invoked with -a, don't retain the anonftp setting between
hosts (`ftp somehost:' sets anonftp, but you don't want that to `stick'
if you close that connection and open a new one).
 1.83 10-Oct-1999  lukem use sigjmp_buf for sigsetjmp(), instead of jmp_buf.
noted by Havard.Eidnes@runit.sintef.no.
 1.82 09-Oct-1999  lukem allow a second SIGINT during the "xfer aborted. waiting for remote to finish abort."
stage. if this occurs, just call lostpeer() to close the connection. whilst this
might be considered brutal, it's also extremely handy if you're impatient or there's
lossage at the remote end.
 1.81 09-Oct-1999  lukem * use sigsetjmp()/siglongjump() instead of setjmp()/longjmp(); the latter
don't save the signal mask on some foreign systems.
* ensure signal handlers don't use stdio and do reset errno if they
don't exit with siglongjmp()
* use a common SIGINT handler for {send,recv}request()
 1.80 05-Oct-1999  lukem * set SIGQUIT to psummary in each of the xfer routines. (editline seems to
override SIGQUIT when EL_SIGNAL = 1 (but we want the latter for all the
other signal support it has).
* more fixes after previous rototill
 1.79 05-Oct-1999  lukem the prior change was a bit too aggressive in factoring out common code in
{send,recv}request(). completion and uploading now works again...
 1.78 05-Oct-1999  lukem * factor out SIGINFO setting into a handler that is always active (but only
prints out info if bytes > 0). only set the handler if SIGINFO is defined
* hijack SIGQUIT to be the same as SIGINFO (foreign ports have this, and it's
annoying to have SIGQUIT dump core on netbsd when it prints info on other
systems)
* in {recv,send}request(), factor a lot of duplicated code out into a
`cleanup' section at the end
* rework shell() a bit
 1.77 05-Oct-1999  lukem add TNFi copyright to all files i've done more than a minor amount of work to...
 1.76 05-Oct-1999  lukem enhancments from Marc Horowitz <marc@mit.edu> to improve connection timeouts:
* implement xsignal_restart(), which only sets the SA_RESTART flag if
specifically requested
* xsignal() is now a wrapper to xsignal_restart(). INFO, USR1, USR2 and WINCH
are restartable, ALRM, INT, PIPE and QUIT are not.
* improve getreply()'s timeout code to take advantage of the above.

other changes:
* improve wording of how globbing works for `classic' URLs (host:path).
suggested by John Refling <johnr@imageworks.com> in relation to PRs
[bin/8519] and [bin/8520]
* always compile in the `edit' command even if NO_EDITCOMPLETE defined.
it's just a no-op in the latter case, which is more consistent to
the users.
* always compile in about: support (i.e, remove NO_ABOUT).
i'm entitled to some vanity in this program...
* clean up some whitespace
 1.75 01-Oct-1999  lukem restart_point is a global; no need for it here
 1.74 01-Oct-1999  lukem If EPSV or EPRT fails, disable epsv4 for the rest of the current connection.
the disabled state can be overridden by toggling epsv4.

(I got sick of the errors about EPSV not being supported on almost
every server I connect to. This way we retain support for epsv4, but
it's not so whiny after the first failure...)
 1.73 01-Oct-1999  lukem prefix the global variables in ftp_var.h with GLOBAL, which defaults
to "extern" if it's not set. define GLOBAL to (empty) in main.c.
this effectively moves all the globals into main.c whilst retaining
namespace access to them in other source files.
(global vars in header files confuse foreign linkers)
 1.72 30-Sep-1999  lukem * In the !NI_NUMERICHOST case (i.e, getaddrinfo() challenged systems), portnum
should be in host order. found/fixed by Matthias Pfaller <leo@dachau.marco.de>
* parse_url(): improve checking of portnum, and add an extra argument to pass
back the parsed portnum to the caller (reduces a bit of code duplication)
* Move the KAME/WIDE copyrights after the BSD/TNFi ones. Since there was
significantly less code added under the former, it's only fair on the latter.
 1.71 30-Sep-1999  lukem in empty(), FD_ZERO(&rmask) not (&cin). (hi christos! :)
fixes abort_remote() when __USE_SELECT is defined.
thanks to simonb@netbsd.org for reporting this bug
 1.70 29-Sep-1999  lukem * consistentnly use memset(a,0,c); there were some cases of memset(a,'\0',c)
* remove explicit extern int h_errno; it's in <netdb.h>
* add <termios.h> back to util.c; it contains struct winsize on some systems
 1.69 28-Sep-1999  lukem remove debugging cruft
 1.68 28-Sep-1999  lukem * add new commands:
lpage page local files
pdir as dir, but through your $PAGER
pls as ls, but through your $PAGER
* implement docase() (a la dotrans() et al) and use appropriately, rototilling
some duplicated code
* globulize(): modify to return a pointer to the strdup()ed result in all cases,
and hack the code that calls it to take this into account
* replace strcpy() and strncpy() with strlcpy()
* put(), getit(): use some aptly named local vars instead of argv[...]
* delint
 1.67 27-Sep-1999  lukem * idle(): rename to idlecmd(). certain linux distributions have an incompatible
prototype for idle() in <unistd.h> (which i thought was against namespace
and sensibility guidelines, but...)
* consistently use xsignal() instead of signal(). we get known behaviour
in all cases (SA_RESTART), which is good for some borken foreign systems.
* remove signal.h from most files; it's unnecessary now
* fetch_url(): use `long chunksize' instead of ssize_t; it's more portable, and
we're setting chunksize with strtol() anyway
* xsignal(): only use SA_RESTART if it exists. SunOS 4.x doesn't have it
but has the inverse (SA_INTERRUPT). the original function i was inspired
from had this support (lib/signal.c, W. Richard Stevens' `UNP 2nd ed Vol 1').
* remove <termios.h> from util.c; it should be unnecessary now
 1.66 24-Sep-1999  lukem * use %lld instead of %qd to print out (long long) vars.
(slightly more portable; e.g, solaris supports this)
* remove some fluff (lint)
 1.65 24-Sep-1999  lukem fix a couple of thinkos in my recent work:
* abort_remote(): replace borken MIN(4,BUFSIZ) with just BUFSIZ; it
should have been MAX(4,BUFSIZ), but it's probably safe to assume that
BUFSIZ is at least 3... (fix from simonb)
* auth_url(): use the correct variable when calculating a buffer size.
 1.64 22-Sep-1999  lukem branches: 1.64.2;
* add support for `xferbuf', which sets both `sndbuf' and `rcvbuf'
* document the above three commands
* rototill the way the sndbuf and rcvbuf work. remove resetsockbufsize()
* use the appropriate socket buffer size as the size of the buffer that
the read()/write() loops use. speeds up things in some cases.
 1.63 22-Sep-1999  lukem replace snprintf() with strlcpy(), strlcat(), or direct assignment
where appropriate. (strlc*() are easier to port to foriegn systems).

XXX; there's still a few snprintf's in the progress meter stuff to convert
 1.62 21-Sep-1999  lukem * protect more of the AF_INET6 stuff with #ifdef INET6 (for portability)
* in the main data moving loops only call the initial gettimeofday() if
rate throttling is enabled (saves a system call per loop when not
throttling).
 1.61 14-Sep-1999  mycroft warn()->warnx() in a couple of places.
 1.60 03-Sep-1999  itojun sync with recent kAME.
- avoid s6_addr{8,16,32} which are nonstanard.
 1.59 01-Sep-1999  itojun ftpd(8): Copy sin6_scope_id from control connection to active data
connection destination, hoping this to help ftpd's behavior with
scoped IPv6 addresses.
I'm not sure if it is the right way, but it is the best way available to us.
LPRT or EPRT command gives no information about which interface (or scope)
to be used for new data connection.

ftp(1): On data connection establishment, warn if scoped address is used.
If peer (ftp daemon) does not handle scoped address, data connection
may not work right.

This seems to be sort of protocol spec hole, not implementation issue.
 1.58 29-Aug-1999  christos make ftp work again with the traditional gethostbyname/getservbyname
interfaces.
 1.57 20-Jul-1999  itojun cleanup EPSV return code checking part.
remove debug fputs() left by mistake.
 1.56 17-Jul-1999  itojun avoid false warnings on 22x reply code checks. previous code was too picky.

From: Wolfgang Rupprecht <wolfgang@wsrcc.com>
 1.55 13-Jul-1999  itojun implement more fallback case for EPSV. BSDI ftpd is very broken
that it returns status 228 against EPSV, where it must return status
of 229.

separate PASV and LPSV processing.

PR: 7976
 1.54 11-Jul-1999  itojun add epsv4 command, which enable/disable the use of EPSV/EPRT.

this is mainly for (hypothetical) ftp server which disconnect clients
that use EPSV/EPRT. I've never seen any ftp server like this, but
epsv4 command may be of use when such an ftp server is found.
 1.53 11-Jul-1999  christos oops, need to declare tos.
 1.52 10-Jul-1999  christos Kludge around non 4.4BSD systems that don't have a length field in struct
sockaddr*.
 1.51 10-Jul-1999  christos remove unused variable
 1.50 03-Jul-1999  itojun free dynamically allocated storage on error.
 1.49 03-Jul-1999  itojun clearify socket/connect loop.
 1.48 02-Jul-1999  itojun add dual-stack (IPv4/v6) support. hope I broke no other part...
 1.47 02-Jul-1999  lukem make a pointers static again (that were made automatic as part of the
xfer rate stuff, but i never completed the changes that didn't need it
set).
fixes a coredump noticed on current-users@ by Chan Yiu Wah <c5666305@hkstar.com>
 1.46 29-Jun-1999  lukem [fear this; more ftp hacking from lukem :-]

features:
 1.45 24-Jun-1999  christos rework empty() to work with both select and poll and abstract it better.
 1.44 02-Jun-1999  lukem * fix gate mode to login as `user@realhost' rather than using PASSERVE;
the latter only seemed to work for TIS Gauntlet and not TIS fwtk.
thanks to simonb@netbsd.org for testing this. fixes [bin/5556].
* if EOF (e.g, ^D) is entered at a username/password/account prompt which
happens to use fgets(), exit the login rather than treating EOF as CR.
* don't use the comma operator where separate statements are valid
* always use snprintf to copy stuff into malloced buffers, just in case
typos creep in and mean that the buffer ends up being overflowed
 1.43 04-May-1999  lukem print the "xxx bare linefeeds" message after the progressmeter. noted by dan@
 1.42 28-Apr-1999  lukem * make parsing of ftp:// urls more RFC 1738 compliant;
- the path is split on `/', and each directory is CWD-ed into separately.
(from [standards/7484] by Alan Barrett <apb@iafrica.com>)
- support a trailing `;type=X' suffix, where X is a,i, or d. (d isn't
implemented, but it is recognised)
- the only non-compliant behaviour is that empty directories sections
(e.g `//') aren't run as `CWD ' - as a lot of ftpds don't like that.
Instead, treat this as a no-op.
* don't support globbing for ftp urls, since that's technically not
RFC compliant.
* fix a couple of man-page nits
 1.41 19-Feb-1999  lukem branches: 1.41.2;
support restart during proxy transfers (the traditional ftp command, not the
http proxy). seems to work with my limited testing (i'm not a big user of
proxy). bug noted by Jorgen Lundman <lundman@argonaut.com> in [bin/5948]
 1.40 24-Jan-1999  lukem * -v enables verbose & progress, -V disables both
* set setvbuf(ttyout, NULL, _IOLBF, 0) and remove a bunch of fflush(ttyout).
* use fwrite() instead of write() for progressmeter (don't intermix stdio
with non stdio ops)
 1.39 05-Jan-1999  lukem Fall back from passive to active if connect() fails. (from openbsd)
 1.38 08-Aug-1998  lukem * implement xsignal(); same semantics as signal() but uses sigaction
with an explicit SA_RESTART. (needed for portability)
* use xsignal() for SIGALRM and SIGINFO handlers
 1.37 03-Aug-1998  lukem features:
* support $no_proxy, which is a comma or space separated list of
host[:port] elements for which proxying is to be disabled.
(asked for by cgd in [bin/5027])
* if $FTPANONPASS is defined, use that as the anon ftp password
(instead of "`whoami`@")
* allow http URL's without a filename as long as an output file
is specified.

other stuff:
* implement parse_url(), which breaks up a URL into its bits, and use.
* simplify url_get() and auto_fetch() to use parse_url() and to not
modify the supplied URL or a copy of it.
* implement xmalloc() and xstrdup(); error-checked malloc()/strdup()
* add more consistency to messages, quoting strings in output as `%s'
 1.36 10-Jul-1998  thorpej Add "sndbuf" and "rcvbuf" commands for setting the socket buffer sizes,
which in turn can allow the use of larger TCP windows. This is a work in
progress; there is not yet support for specifying global defaults or
user prefrences on a host/network basis.
 1.35 04-Jun-1998  lukem some fixes & enhancements from openbsd's ftp, with extra fixes by me:
* default to passive with active fallback. $FTPMODE modifies this behaviour.
-A forces active connection.
* support '-o outfile' for auto-fetched files. outfile can be a file,
`-' (for stdout), or '|command' (to output each file through command).
* support '-r waittime', which retries the connection after waittime seconds
if it fails.
* fix 'page file' when restart is non-zero.
* try all ip-addresses of a host in a http fetch (as the normal ftp fetch
does).

XXX: a ``broken pipe'' error sometimes occurs with -o '|command';
i haven't tracked this down yet.
 1.34 20-May-1998  pk Restore `preserve' value when we no longer need the modified version.
 1.33 20-May-1998  christos - add <signal.h> since we are using signals.
- add <sys/time.h> since we are using utimes
- don't require quad_t to exist to compile.
 1.32 01-Apr-1998  kleink Need <time.h> for asctime() and localtime() prototypes.
 1.31 18-Jan-1998  lukem * ensure buffer for username is initialised, so ^D on username prompt
doesn't use garbage for the username. from "Soren S. Jorvang" <soren@t.dk>
in [bin/4559]
* use in_port_t for ports, and USHRT_MAX instead of 0xffff
(from millert@openbsd.org)
* use `NULL' instead of `(.... *)0' where appropriate.
 1.30 01-Nov-1997  lukem * in recvrequest(), ignore restart_point unless "RETR"ieving. fixes problems
where a remote completion or `mget' would confuse the client a `restart'
had been issued beforehand. now, `restart' is remembered until an operation
that can actually use it is invoked.
* in sendrequest(), don't reset restart_point upon entry. fixes `restart'
for `put' operations.
* if `restart' is invoked with no arguments, print current setting instead
of displaying a usage
* consistently use printf("%qd", (long long)restart_point) when displaying
restart_point
* use strto[lq]() instead of atol() when parsing `mark' and `restart' values
* remove unnecessary strlen()s when result of previous snprintf() will do
* replace a few malloc()/strcpy()s with strdup()s
* use SECSPERHOUR instead of '3600'
 1.29 19-Oct-1997  mycroft branches: 1.29.2;
Use S_IS*(), not S_IF*.
 1.28 13-Sep-1997  lukem Fixes from Todd Miller <Todd.Miller@courtesan.com>:
* use size_t instead of int in places
* use symbolic constants when using access()
 1.27 18-Aug-1997  lukem bugs fixed:
* don't interpret '-' or '|' when a local filename is determined from
the remote name (i.e, in mget, and in get with only one argument).
This is implemented using an extra argument to recvrequest().
Fixes a major security hole.
* clean up memory leak when using globulize()
* clean up a couple of comments
* fix wording in TNF copyright

features added:
* support for TIS fwtk gate-ftp servers:
* read defaults from $FTPSERVER && $FTPSERVERPORT
* start in gate-ftp mode if invoked as 'gate-ftp'
* toggle or set with 'gate [host [port]]'
 1.26 20-Jul-1997  lukem * use RCSID() && COPYRIGHT() macros
* cleanup for WARNS=1 (including some ugly '(void)&var' bits wrapped in
#ifdef __GNUC__ to shut up gcc warnings WRT setjmp/longjmp)
* use strtol() instead of atoi(), and more extensively check result of
conversion
* use u_int16_t instead of short or int for TCP port addresses
 1.25 14-Apr-1997  lukem More enhancements/bugfixes (when will it end?)
* differentiate between being connected, and being logged in
* cleanup some text messages
* support username & password ftp URLs (ftp://user:pass@host/) in non-proxy
situations; assume proxy supports it for proxy situations.
* cd to / before performing any autofetch transfers
* use strncasecmp in URL parsing. fix from <Todd.Miller@courtesan.com>
 1.24 16-Mar-1997  lukem Fixes from <Todd.Miller@cs.colorado.edu>, with some cleanup and reworks by me:
* only echo "Passive mode" in verbose mode; scripts that use ftp
may get unwanted output otherwise
* disable progress bar and modification time preservation when
retreiving to a non-regular files. fixes progress bar getting in
way of "get file /dev/tty"
* setup el_init() et al if editing is set, not if fromatty.
TODO: migrate this to a function, and call if editing is turned on later
in the session. also implement edit_cleanup if editing is turned off
* call el_set() after setting SIGWINCH handler. This fixes the problem
when suspending in a non-cbreak shell (e.g, csh) would trash your tty mode.
* reset interactive mode correctly in auto_fetch() mget mode
 1.23 13-Mar-1997  lukem Features:
* support remglobbing of auto_fetch arguments
* new flag - '-e'; disable editing
* "page file" == "get file |${PAGER-less}"

Bugfixes/cleanup:
* consistently use a trailing '.' on messages
* code cleanup, including buffer overrun fixes, use puts
and putchar in places, etc (inspired by OpenBSD mods)
* disable progress bar when local-file is a pipe or '-'
* skip \r in http headers
* fix remote ftpd slash bug more elegantly (so it works with ////)
* abort_remote(): check if cout==NULL before using it. should fix [bin/3273]
* fixed up cosmetic problems when complete_remote() generated errors from the
remote server (such as "no files found", "login with user and pass", ...)
done by adding extra argument to remglob(), which is a pointer to an error
buffer to put messages in rather than printing to stdout.
 1.22 01-Feb-1997  lukem [Yet Another Huge Ftp Commit - hopefully the last for a while,
barring any more little things people want added ...]

New features:
* progressmeter is now asynchronous, so "stalled" transfers can be
detected. "- stalled -" is displayed instead of the ETA in this case.
When the xfer resumes, the time that the xfer was stalled for is
factored out of the ETA. It is debatable whether this is better than
not factoring it out, but I like it this way (I.e, if it stalls for 8
seconds and the ETA was 30 seconds, when it resumes the ETA will still
be 30 seconds).
* verbosity can be disabled on the command line (-V), so that in auto-fetch
mode the only lines displayed will be a description of the file, and
the progress bar (if possible)
* if the screen is resized (and detected via the SIGWINCH signal), the
progress bar will rescale automatically.

Bugs fixed:
* progress bar will not use the last character on the line, as this can
cause problems on some terminals
* screen dimensions (via ioctl(TIOCWINSZ)) should use stdout not stdin
* progressmeter() used some vars before initialising them
* ^D will quit now. [fixes bin/3162]
* use hstrerror() to generate error message for host name lookup failure.
* use getcwd instead of getwd (it should have been OK, but why tempt fate?)
* auto-fetch transfers will always return a positive exit value upon failure
or interruption, relative to the file's position in argv[].
* remote completion of / will work, without putting a leading "///".
This is actually a bug in ftpd(1), where "NLST /" prefixes all names
with "//", but fixing every ftpd(1) is not an option...
 1.21 19-Jan-1997  lukem New features:
* Command line editing via editline(3) library.
* Context sensitive command and file completion, including remote files.

Enhancements to auto-fetch feature:
* Support for http:// URLs using the http protocol, including proxy HTTP
support via $htty_proxy if it's defined.
* The connection is kept open between successive files on the same host.
(obviously, this does not count for http requests.)
* Return value of ftp is 0 on no error, or the offset in argv[] of the file
which failed (i.e., argv[x] failed, ftp returns x).
* If the path in an ftp URL or classic format line has a trailing '/',
cd to the path and enter interactive mode. Fixes [bin/3011], albiet
requiring the user to help ftp in determining the operation.

Other changes:
* '-P port' works for normal ftp, and is the default for all classic style
auto-fetch transfers and for ftp URLs that don't specify the port.
(previously it would just work for the first xfer.)
* Some code moved into separated files along logical divisions.
* Editing and completion can be compiled out with -DSMALLFTP.
 1.20 09-Jan-1997  tls RCS ID police
 1.19 29-Dec-1996  lukem * preserve modtime if size is 0 [bin/3040, Enami Tsugutomo]
* in autofetch mode, don't retry a file if the connection failed - skip it
and move to the next file. [bin/3051, Matt Green]
* only print error messages from SIZE or MDTM if the user used 'size' or
'modt' (respectively). prevents extraneous warnings during normal transfers
when connected to a site which doesn't support these (behaviour prior to
last commit), but still allows error feedback to specific user requests
for this info (which the last commit broke).
* 'account': only accept one optional argument
* if invoked as pftp, default to passive mode on (from FreeBSD)
* remove leading '0 ' in progress bar - looked ugly
* use warn instead of perror
* use strncpy when src isn't known to have safe length
* remglob(): use mkstemp() to prevent symlink games, and don't override
_PATH_TMP, use it as the prefix to the temp file
 1.18 25-Dec-1996  christos - make sure that the reply string is null-terminated, and copy only up to
the initialized copy of the source string, since the source string is not
null terminated at this point.
 1.17 06-Dec-1996  lukem 'b' == bits not bytes. for all prefixes >= 'K', explicitly specify 'B'
afterwards. don't print anything for a pure byte count. rework ETA display
as well.
 1.16 06-Dec-1996  lukem functionality mods:
* implement 'progress bar/meter' (inspired by ncftp). use 'progress' to
toggle on. it will display current file size to 5 digits, automatically
determining suffix (up to 16384 P (petabytes) == 2^64).
* 'ls' now uses NLST (unadorned listing), a la older ftp clients. 'dir'
still does LIST (long listing). idea from John Nemeth <jnemeth@cue.bc.ca>

bug fixes:
* return first line of reply in reply_string[] from getreply(), instead
of last line. This fixes [bin/741] (parsing of SYST), and also means
that SIZE and MDTM messages will be parsed correctly if they're longer
than 1 line.
* parse URL-style auto-ftps that have no filename correctly
(e.g, ftp://host, ftp://host/, ftp://host/dir/). pointed out by
Jaromir Dolecek <dolecek@saruman.ics.muni.cz>
* pass the correct size array in 2nd arg of utimes() when setting the
modification time
 1.15 28-Nov-1996  lukem More features, some of which were inspired by changes that
friedman@gnu.ai.mit.edu (Noah Friedman) made to his modified ftp:
- implement "lpwd" - local pwd
- implement "preserve" - toggle preserving of file modification
times on retrieved files
- allow for explicit "on" or "off" arg to toggle commands
- "exit" synonym for "quit", "msend" synonym for "mput"
- in confirmation mode, allow 'a' (yes to rest of current command),
and 'p' (turn off prompt mode, as if 'prompt off' was done,
effective immediately)
- "modtime" returns time formatted as localtime, not GMT

Bug fixes:
- check for extraneous args on commands
- cleanup const usage, line formatting
- create 0 length temporary file in remglob() to prevent symlink games
(from OpenBSD)
- check length of filename of ~/.netrc (from OpenBSD)
 1.14 25-Nov-1996  lukem cleanups/bugfixes:
- don't echo 'ACCT' parameter when debugging (a la 'PASS')
- Fix checking of directory access for "/foo", the parent
directory is "/", not "" (from FreeBSD)
- remove trailing whitespace on lines
- add any missing NetBSD tags
- cleanups to man page, includinging sorting options description

feature additions:
- variable sized hash marks (from [bin/683], but done in the hash command
as an optional arg)
- more user-friendly transfer time printing (from FreeBSD, with mods)
- '-p' command line option to jump into PASV mode (closes [bin/2857],
but with an option rather than checking argv[0])
- SIGINFO support for printing xfer stats when sending/receiving requests
- '-P port' for changing the port to connect to (from thorpej@netbsd.org)
- '-a': bypass normal login, and try anonymous login (from OpenBSD
via thorpej)
- autofetch files via url (ftp://...) or "classic" (host:/file)
(from OpenBSD via thorpej)
- 'ftp' synonymous with 'open' (from FreeBSD)
 1.13 16-Sep-1995  pk Correct timersub() argument order (from Thomas Eberhardt; PR#1470).
 1.12 08-Sep-1995  tls Sync with 4.4lite2
 1.11 21-May-1995  mycroft Use inet_aton(), not inet_addr().
 1.10 21-Mar-1995  mycroft Update to use timer{add,sub}().
 1.9 29-Aug-1994  mycroft branches: 1.9.2;
Add RCS ids.
 1.8 25-Aug-1994  cgd passive mode support, from David Carrel <carrel@cisco.com>, and cleaned
up for the new ftp sources by me.
 1.7 25-Aug-1994  cgd the previous local changes
 1.6 25-Aug-1994  cgd clean up import, no local changes.
 1.5 28-Mar-1994  cgd kill typo
 1.4 27-Mar-1994  cgd off_t foo
 1.3 23-Sep-1993  mycroft Display bytes/second rather than kbytes/sec, and use 3 digits of
precision for transfer time.
 1.2 01-Aug-1993  mycroft Add RCS identifiers.
 1.1 21-Mar-1993  cgd branches: 1.1.1; 1.1.2;
Initial revision
 1.1.2.1 25-Aug-1994  cgd import from 4.4-Lite
 1.1.1.2 05-Sep-1995  tls imported from 44lite2
 1.1.1.1 21-Mar-1993  cgd initial import of 386bsd-0.1 sources
 1.9.2.2 29-Aug-1994  mycroft Add RCS ids.
 1.9.2.1 29-Aug-1994  mycroft file ftp.c was added on branch netbsd-1-0 on 1994-08-29 03:09:16 +0000
 1.29.2.2 10-Nov-1998  cgd pull up rev(s) 1.31-1.38 from trunk. (feyrer)
 1.29.2.1 18-Nov-1997  mellon Pull rev 1.30 up from trunk (lukem)
 1.41.2.1 22-Jun-1999  perry pullup 1.41->1.44 (lukem)
 1.64.2.1 27-Dec-1999  wrstuden Pull up to last week's -current.
 1.95.2.1 23-Jun-2000  minoura Sync w/ netbsd-1-5-base.
 1.100.2.2 26-Apr-2002  he Pull up revision 1.118 (requested by itojun):
Avoid buffer overrun on PASV response from a malicious server.
 1.100.2.1 18-Oct-2000  tv Pullup usr.bin string format fixes [is].
See "cvs log" for explicit revision numbers per file, from sommerfeld.
 1.126.2.9 24-Jul-2005  tron Pull up revision 1.135 (requested by lukem in ticket #606):
Add NO_USAGE and NO_DEBUG so that we can fit in the floppies again.
 1.126.2.8 24-Jul-2005  tron Pull up revision 1.134 (requested by lukem in ticket #606):
Implement:
int getline(FILE *stream, char *buf, size_t buflen, const char **errormsg)
Read a line from the FILE stream into buf/buflen using fgets(), so up
to buflen-1 chars will be read and the result will be NUL terminated.
If the line has a trailing newline it will be removed.
If the line is too long, excess characters will be read until
newline/EOF/error.
Various -ve return values indicate different errors, and errormsg
will be changed to an error description if it's not NULL.
Convert to use getline() instead of fgets() whenever reading user input
to ensure that an overly long input line doesn't leave excess characters
for the next input operation to accidentally use as input.
Zero out the password & account after we've finished with it.
Consistently use getpass(3) (i.e, character echo suppressed) when
reading the account data. For some reason, historically the "login"
code suppressed echo for Account: yet the "user" command did not!
Display the hostname in the "getaddrinfo failed" warning.
Appease some -Wcast-qual warnings. Fixing all of these requires
significant code refactoring. (mmm, legacy code).
 1.126.2.7 24-Jul-2005  tron Pull up revision 1.133 (requested by lukem in ticket #606):
* Only print the "Trying <address>..." message if verbose and
there's more than one struct addrinfo in the getaddrinfo() result.
* Don't use non-standard "u_int".
 1.126.2.6 24-Jul-2005  tron Pull up revision 1.132 (requested by lukem in ticket #606):
Fix some cast issues highlighted by Scott Reynolds using gcc 4 on OSX.4
 1.126.2.5 18-May-2005  snj Pull up revision 1.131 (requested by lukem in ticket #301):
* Correct the "optlen" argument passed to getsockopt(3) and setsockopt(3)
in various places. Fixes a problem noted by Allen Briggs.
* Improve warning printed when connect(2) for the data channel fails.
 1.126.2.4 18-May-2005  snj Pull up revision 1.130 (requested by lukem in ticket #318):
Use socklen_t instead of int as the 5th argument to getsockopt().
Improve invocation of setsockopt() and associated failure messages.
 1.126.2.3 09-May-2005  tron Pull up revision 1.129 (requested by lukem in ticket #266):
typo in previous
 1.126.2.2 09-May-2005  tron Pull up revision 1.128 (requested by lukem in ticket #266):
gratuitous whitespace cleanup (before someone else jumps the gun...)
 1.126.2.1 09-May-2005  tron Pull up revision 1.127 (requested by lukem in ticket #265):
Implement a timeout on the accept(2) in dataconn() and the
connect(2) in xconnect() by temporarily setting O_NONBLOCK
on the socket and using xpoll() to wait for the operation
to succeed.
The timeout used is the '-q quittime' argument (defaults to
60s for accept(2), and the system default for connect(2)).
Idea inspired by discussion with Chuck Cranor.
This may (indirectly) fix various problems with timeouts
in active mode through broken firewalls.
Implement xpoll() as a wrapper around poll(2), to make it
easier to replace on systems without a functional poll(2).
Unconditionally use xpoll() instead of conditionally using
select(2) or poll(2).
 1.152.6.2 22-Jul-2007  lukem Replace HAVE_SOCKADDR_SA_LEN with defined(HAVE_STRUCT_SOCKADDR_SA_LEN)
 1.152.6.1 22-Jul-2007  lukem file ftp.c was added on branch matt-mips64 on 2007-07-22 05:02:51 +0000
 1.152.4.1 09-Jan-2008  matt sync with HEAD
 1.153.6.1 18-May-2008  yamt sync with head.
 1.155.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.156.6.1 13-May-2009  jym Sync with HEAD.

Third (and last) commit. See http://mail-index.netbsd.org/source-changes/2009/05/13/msg221222.html
 1.162.2.2 30-Oct-2012  yamt sync with head
 1.162.2.1 17-Apr-2012  yamt sync with head
 1.163.2.2 27-Aug-2016  bouyer Apply patch, requested by nonaka in ticket #1375:
src/usr.bin/ftp/cmds.c: patch
src/usr.bin/ftp/fetch.c: patch
src/usr.bin/ftp/ftp.1: patch
src/usr.bin/ftp/ftp.c: patch
src/usr.bin/ftp/ftp_var.h: patch
src/usr.bin/ftp/main.c: patch
src/usr.bin/ftp/ssl.c: patch
src/usr.bin/ftp/ssl.h: patch
src/usr.bin/ftp/version.h: patch
Update ftp(1) to version 20150912, adding https via proxy support.
 1.163.2.1 17-Dec-2013  bouyer Apply patch, requested by tron in ticket #997:
usr.bin/ftp/Makefile patch
usr.bin/ftp/cmds.c patch
usr.bin/ftp/cmdtab.c patch
usr.bin/ftp/extern.h patch
usr.bin/ftp/fetch.c patch
usr.bin/ftp/ftp.1 patch
usr.bin/ftp/ftp.c patch
usr.bin/ftp/ftp_var.h patch
usr.bin/ftp/main.c patch
usr.bin/ftp/progressbar.c patch
usr.bin/ftp/ssl.c patch
usr.bin/ftp/ssl.h patch
usr.bin/ftp/util.c patch
usr.bin/ftp/version.h patch

Add HTTPS support to ftp(1).
 1.164.10.1 13-Mar-2016  martin Pull up following revision(s) (requested by nonakap in ticket #1133):
usr.bin/ftp/fetch.c: revision 1.208-1.221
usr.bin/ftp/cmds.c: revision 1.136-1.137
usr.bin/ftp/ssl.c: revision 1.5
usr.bin/ftp/ftp.c: revision 1.165-1.166
usr.bin/ftp/ftp_var.h: revision 1.84

Workaround const issues of SSL_set_tlsext_host_name.

Use the proper format "[IPv6 address]:port" when reporting connection
attempts to IPv6 endpoints.

(Hopefully) fix build without IPv6 support

Try to factor out some code, this is completely out of control.

Separate no_proxy handling.

Factor the proxy handling code out.

Fix compile failure without WITH_SSL.

PR/50438: NONAKA Kimihiro: ftp(1): CONNECT method support

make DPRINTF/DWARN always statements.

Fix to connect https via proxy.

Fix ttyout message.

Simplify and factor out connect message

Split the position/size parsing into a separate function.

Mark function as only needed with ssl.

Fix downloads of local files using file:// URLs

Initialize the token match pointer.

use sizeof() and array notation.
CID 1354295: Array overrun.
 1.166.2.1 04-Nov-2016  pgoyette Sync with HEAD
 1.167.14.1 10-Jun-2019  christos Sync with HEAD
 1.167.6.3 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1763:

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/cmds.c up to 1.141
usr.bin/ftp/complete.c up to 1.47
usr.bin/ftp/domacro.c up to 1.23
usr.bin/ftp/extern.h up to 1.82
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp.c up to 1.174
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/progressbar.c up to 1.24
usr.bin/ftp/progressbar.h up to 1.9
usr.bin/ftp/ssl.c up to 1.12
usr.bin/ftp/ssl.h up to 1.5
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

ftp(1): validate address from PASV and LPSV response.
ftp(1): use raw write(2) instead of fwrite(3) to avoid stream
corruption because of the progress bar interrupts.
Fixes for PR 56219 and PR 55857.
PR 57003: Support relative redirects.
 1.167.6.2 12-Sep-2022  martin Backout ticket #1763 for now - trust anchors are not solved.
 1.167.6.1 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1763:

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/cmds.c up to 1.141
usr.bin/ftp/complete.c up to 1.47
usr.bin/ftp/domacro.c up to 1.23
usr.bin/ftp/extern.h up to 1.82
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp.c up to 1.174
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/progressbar.c up to 1.24
usr.bin/ftp/progressbar.h up to 1.9
usr.bin/ftp/ssl.c up to 1.11
usr.bin/ftp/ssl.h up to 1.5
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

ftp(1): validate address from PASV and LPSV response.
ftp(1): use raw write(2) instead of fwrite(3) to avoid stream
corruption because of the progress bar interrupts.
Fixes for PR 56219 and PR 55857.
PR 57003: Support relative redirects.
 1.168.2.7 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1523

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/ssl.c up to 1.12
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

PR 57003: Support relative redirects.
 1.168.2.6 24-Oct-2021  martin Pull up following revision(s) (requested by lukem in ticket #1367):

usr.bin/ftp/ftp.c: revision 1.174

ftp: remove unnecessary variable assignments

Remove assignment to error in initconn(); it's not tested anywhere after the
initial use, so no need to set it before goto bad.
(Looks like copypasta from the initial addition of the code in rev 1.48.)
 1.168.2.5 24-Oct-2021  martin Pull up following revision(s) (requested by lukem in ticket #1366):

usr.bin/ftp/ftp.c: revision 1.173

ftp: validate address from PASV and LPSV response

Fail if the server's response to PASV or LPSV contains an IP address
that doesn't match that of the control connection.
(EPSV already only uses the port portion of the server's response,
per RFC 2428).

Previously a hostile server could cause ftp to open a data connection elsewhere.
Many other ftp implementations have had a similar change for many years,
including those in popular browsers (before they deprecated FTP ...)

Thanks to Simon Josefsson notifying me about
https://lists.gnu.org/archive/html/bug-inetutils/2021-06/msg00002.html
 1.168.2.4 14-Jun-2021  martin Pull up following revision(s) (requested by lukem in ticket #1293):

usr.bin/ftp/ftp.c: revision 1.172

set SO_KEEPALIVE on control connection

Attempt to prevent timeouts of the control connection by setting SO_KEEPALIVE.
This matches the equivalent behaviour in ftpd.

Note: This is a much simpler change than adding a background polling event
to invoke "STAT" (or "NOOP") on the control connection during a transfer.
(It's unclear from RFC 959 whether "NOOP" is even permitted during a transfer).

PR bin/56129
 1.168.2.3 14-Jun-2021  martin Pull up following revision(s) (requested by lukem in ticket #1291):

usr.bin/ftp/ftp.c: revision 1.169
usr.bin/ftp/util.c: revision 1.161

ftp: exit if lostpeer invoked by a signal

lostpeer() calls too many async-unsafe functions (both directly
and indirectly) to close and cleanup the remote connections,
so just exit after the cleanup if invoked by a signal.

Reported in private mail by Qi Hou.

May also resolve a crash reported by Thomas Klausner.
 1.168.2.2 14-Jun-2021  martin Pull up following revision(s) (requested by lukem in ticket #1290):

usr.bin/ftp/version.h: revision 1.90
usr.bin/ftp/ftp.c: revision 1.170

ftp.c: improve signal handler restoration

Only invoke the old signal handler if it's a real signal handler
and not SIG_IGN, SIG_DFL, SIG_HOLD, or SIG_ERR, using new static
function issighandler().

Avoids an intermittent race condition with a null pointer
dereference via (*SIG_DFL)().

Bug class reported by Joyu Liao from Juniper Networks.

Use SIG_ERR instead of NULL as the indicator that a signal handler
hasn't been changed, so that SIG_DFL (equivalent to NULL)
will be restored.
 1.168.2.1 29-Jan-2021  martin Pull up following revision(s) (requested by lukem in ticket #1190):

usr.bin/ftp/progressbar.c: revision 1.24
usr.bin/ftp/ssl.c: revision 1.9
usr.bin/ftp/progressbar.h: revision 1.9
usr.bin/ftp/ftp.c: revision 1.171
usr.bin/ftp/version.h: revision 1.92

ftp: don't use restartable signals

Refactor to not rely upon restartable signals (SA_RESTART),
possibly fixing intermittent failures with -q QUITTIME.
ftp transfers: handle EINTR/EAGAIN in copy_bytes(),
instead of relying upon restartable signals.

http/https transfers: Explicitly print an error similar to
progressmeter() when timing-out for -Q QUITTIME in fetch_wait(),
and set errno to ETIMEDOUT so that the warn() in fetch_url()
prints a more accurate error message.

PR/55857
 1.171.4.1 06-Jun-2021  cjep sync with head
 1.174.2.3 02-Dec-2024  martin Pull up following revision(s) (requested by lukem in ticket #1021):

usr.bin/ftp/ftp.c: revision 1.178
usr.bin/ftp/version.h: revision 1.99
usr.bin/ftp/ruserpass.c: revision 1.35
usr.bin/ftp/main.c: revision 1.134
usr.bin/ftp/main.c: revision 1.135
usr.bin/ftp/progressbar.c: revision 1.27
usr.bin/ftp/util.c: revision 1.165
usr.bin/ftp/cmds.c: revision 1.144
usr.bin/ftp/extern.h: revision 1.84
usr.bin/ftp/fetch.c: revision 1.242
usr.bin/ftp/ftp.1: revision 1.160

s/bninary/binary/ in comment.

extract duplicate code into a function.

Check bounds when copying to destination.

Remove const where the const string ended up being overwritten.

use unsigned when doing shifts.

remove const when string gets overwritten.

ftp: exit non-zero if short http transfer when filesize is known
If a http file size is known and the fetch finishes with less bytes
transferred, exit non-zero.
Bump version to 20241129.
PR bin/54713
PR bin/58281

ftp: help improvements
Document -? as a separate mode.
Document -H HEADER in the usage.
Clarify units for -b and -x.
Consistent argument names between ftp -? and ftp(1).

ftp: order getopt Upper before lower
Consistently order options in getopt and the switch
with the upper case option before the lower case option.
This makes it easier to cross-reference with -? and ftp(1).
No functional change.
 1.174.2.2 13-Oct-2024  martin Pull up following revision(s) (requested by riastradh in ticket #970):

tests/usr.bin/Makefile: revision 1.42
usr.bin/ftp/ruserpass.c: revision 1.34
usr.bin/ftp/main.c: revision 1.130
usr.bin/ftp/ssl.c: revision 1.17
usr.bin/ftp/main.c: revision 1.131
usr.bin/ftp/ssl.c: revision 1.18
usr.bin/ftp/main.c: revision 1.132
usr.bin/ftp/ssl.c: revision 1.19
usr.bin/ftp/main.c: revision 1.133
distrib/sets/lists/tests/mi: revision 1.1342
usr.bin/ftp/ftp.1: revision 1.151
usr.bin/ftp/ftp.1: revision 1.152
usr.bin/ftp/progressbar.c: revision 1.25
usr.bin/ftp/ftp.1: revision 1.153
usr.bin/ftp/progressbar.c: revision 1.26
usr.bin/ftp/ftp.1: revision 1.155
usr.bin/ftp/ftp.1: revision 1.156
usr.bin/ftp/fetch.c: revision 1.239
usr.bin/ftp/ftp.1: revision 1.157
usr.bin/ftp/ftp.1: revision 1.158
usr.bin/ftp/ftp.1: revision 1.159
usr.bin/ftp/ftp_var.h: revision 1.87
etc/mtree/NetBSD.dist.tests: revision 1.208
usr.bin/ftp/ftp_var.h: revision 1.88
usr.bin/ftp/ftp_var.h: revision 1.89
usr.bin/ftp/cmds.c: revision 1.142
usr.bin/ftp/util.c: revision 1.168
usr.bin/ftp/cmds.c: revision 1.143
tests/usr.bin/ftp/custom_headers.sh: revision 1.1
usr.bin/ftp/ssl.c: revision 1.20
usr.bin/ftp/complete.c: revision 1.48
tests/usr.bin/ftp/Makefile: revision 1.1
tests/usr.bin/ftp/t_custom_headers.sh: revision 1.1
usr.bin/ftp/fetch.c: revision 1.240
usr.bin/ftp/fetch.c: revision 1.241
usr.bin/ftp/ftp.c: revision 1.176
usr.bin/ftp/ftp.c: revision 1.177
(all via patch)

ftp(1): wording and formatting improvements

Fix grammar issue with "Support values" reported in private mail.
Document all file transfer types in "type" and cross-reference that.
Consistency fixes in describing file transfer parameters and types.

Fix some mandoc -Tlint issues (except "useless macro: Tn").

Add -b <buflen> to specify the buffer size.

ftp: bump FTPBUFLEN from 4kB to 16kB
sourceforge.net returns a 5kB content-security-policy.
Analyzed by mlelstv@ who reports usual limits are between 4kB and 48kB.
default is now 16K

ftp: improve -b documentation

Order -b bufsize in the synopsis.

Document the actual default value.

ftp: improve units used in comments and errors
Use "KiB" instead of "K" in errors.
Clarify related comments.

pass some lint.

PR/58581: Sunil Nimmagadda: Add flag to allow specifying extra http header
fields.

ftp(1): Nix trailing whitespace in man page.
No functional change intended.

PR bin/58581: ftp(1) should allow specifying header fields in http requests
fix markup (h -> H), explain about multiple headers, fix usage (from RVP)

Don't forget the dot, use the intended macro name (I think),
and improve the wording a little. (All related to the -H option.)
ftp(1): Add test for custom HTTP header fields.

Based on a patch from Sunil Nimmagadda.

PR bin/58581: ftp(1) should allow specifying header fields in http
requests
 1.174.2.1 16-May-2023  martin Pull up following revision(s) (requested by lukem in ticket #171):

usr.bin/ftp/ssl.c: revision 1.15
usr.bin/ftp/util.c: revision 1.167
usr.bin/ftp/ftp.c: revision 1.175
usr.bin/ftp/version.h: revision 1.97

add timeout for ssl connect

Implement a timeout for SSL connection setup, using -q QUITTIME,
defaulting to 60 seconds.

SSL_connect(3) (unlike connect(2)) doesn't timeout by default.
Adapt ssl error messages destination: if unexpected error
from local API, use warn()/warnx() to stderr;
if expected error from a network operation (e.g., timeouts),
use fprintf to ttyout (which might be stdout).

Consistently use ftp_poll() instead of select();
ssl.c (using select()) was added 7 years after the
previous uses of select() were converted to poll().

Check EAGAIN as well as existing EINTR error from ftp_poll(),
for portability.
 1.176.2.1 02-Aug-2025  perseant Sync with HEAD
 1.89 25-Sep-2024  christos PR/58581: Sunil Nimmagadda: Add flag to allow specifying extra http header
fields.
 1.88 18-Feb-2024  wiz branches: 1.88.2;
ftp: bump FTPBUFLEN from 4kB to 16kB

sourceforge.net returns a 5kB content-security-policy.
Analyzed by mlelstv@ who reports usual limits are between 4kB and 48kB.
 1.87 18-Feb-2024  christos Add -b <buflen> to specify the buffer size.
 1.86 10-Sep-2021  rillig branches: 1.86.2;
usr.bin: remove unnecessary lint comment CONSTCOND

Since 2021-01-31, lint no longer warns about 'do ... while (0)'.

No functional change.
 1.85 20-Nov-2017  kre branches: 1.85.6;
Issue PWD commands to the server only when we actually
need the results, not speculatively, just in case we might.

Allows operation with some broken servers that get confused
by PWD commands in some situations, and saves server round
trips in the (modern) common case of
ftp ftp://path/name
where we never need to know the results from PWD.
 1.84 16-Dec-2015  christos branches: 1.84.8;
make DPRINTF/DWARN always statements.
 1.83 12-Jan-2015  christos Increase the buffer limit; otherwise files in:

http://www.taxdetective.ca/Samples/sampledatafiles.html

fail.
 1.82 21-Dec-2012  christos branches: 1.82.8;
PR/47276: Add https support
 1.81 12-Apr-2009  lukem branches: 1.81.6; 1.81.8; 1.81.12;
Fix numerous WARNS=4 issues (-Wcast-qual -Wsign-compare).
 1.80 30-Sep-2008  lukem branches: 1.80.6;
update copyrights
 1.79 10-May-2008  skd Add epsv6 and epsv to disable extended passive mode for ipv6 or both ipv4 and ipv6 respectively. This hack is due to our friends a Juniper Networks who break
epsv in ipv6. Should be fixed in ScreenOS 6.2.X.
 1.78 28-Apr-2008  martin branches: 1.78.2;
Remove clause 3 and 4 from TNF licenses
 1.77 05-Dec-2007  lukem branches: 1.77.6;
DPRINTF() consistency tweaks
 1.76 05-Dec-2007  lukem Rename HAVE_STRUCT_SOCKADDR_SA_LEN to HAVE_STRUCT_SOCKADDR_IN_SIN_LEN
to accurately reflect the structure member being used.
 1.75 22-Jul-2007  lukem branches: 1.75.4; 1.75.6;
Replace HAVE_SOCKADDR_SA_LEN with defined(HAVE_STRUCT_SOCKADDR_SA_LEN)
 1.74 17-Apr-2007  lukem * Implement -s srcaddr; uses srcaddr as the local IP address for all
connections.
Based on code in the version of ftp that FreeBSD had before they
replaced it with lukemftp.
* Move error message handling into ftp_connect() rather than in the
caller, so that more specific error reporting can occur.
* Improve consistency of various warning and error messages.
 1.73 31-Jan-2006  christos rename debug to ftp_debug. grr libssh.
 1.72 29-Jun-2005  christos Add NO_USAGE and NO_DEBUG so that we can fit in the floppies again.
 1.71 11-Apr-2005  lukem gratuitous whitespace cleanup (before someone else jumps the gun...)
 1.70 11-Apr-2005  lukem Implement a timeout on the accept(2) in dataconn() and the
connect(2) in xconnect() by temporarily setting O_NONBLOCK
on the socket and using xpoll() to wait for the operation
to succeed.
The timeout used is the '-q quittime' argument (defaults to
60s for accept(2), and the system default for connect(2)).
Idea inspired by discussion with Chuck Cranor.
This may (indirectly) fix various problems with timeouts
in active mode through broken firewalls.

Implement xpoll() as a wrapper around poll(2), to make it
easier to replace on systems without a functional poll(2).
Unconditionally use xpoll() instead of conditionally using
select(2) or poll(2).
 1.69 03-Jan-2005  lukem branches: 1.69.2;
Forbid filenames returned from mget that aren't in (or below) the
current directory.
The previous behaviour (of trusting the remote server's response when
retrieving the list of files to mget with prompting disabled) has been
in ftp ~forever, and has been a "known issue" for a long time.
Recently an advisory was published by D.J. Bernstein on behalf of
Yosef Klein warning of the problems with the previous behaviour, so
to alleviate concern I've fixed this with a sledgehammer.

Remember the local cwd after any operation which may change it.
Use "remotecwd" instead of "remotepwd".
 1.68 21-Jul-2004  lukem Slightly rework SIGINT handling; if we're exiting the auto-fetch stuff
and sigint_raised is non-zero, reset the handler for SIGINT to SIG_DFL
and raise(SIGINT) so that the appropriate wait(3) status is setup.
Based on solution proposed by Ognyan Kulev.
This should really fix PR [pkg/26351].
 1.67 20-Jul-2004  lukem If an ftp auto-fetch transfer is interrupted by SIGINT (usually ^C),
exit with 130 instead of 1 (or rarely, 0).
This allows an ftp auto-fetch in a shell loop to correctly terminate the loop.
Should fix PR [pkg/26351], and possibly others.
 1.66 06-Jun-2004  christos Save approximately 8K by not including http authentication, extended status
messages and help strings when the appropriate options are set.
 1.65 07-Aug-2003  agc Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22365, verified by myself.
 1.64 21-Jan-2003  jhawk Abstract out progress bar support into progressbar.[ch], for inclusion
in external programs (conditionalized on -DSTANDALONE_PROGRESS).

The following moved from util.c to progressbar.c:
alarmtimer(), progressmeter(), psummary(), ptransfer(),
xsignal(), xsignal_restart()

The following moved from extern.h and ftp_var.h to progressbar.h:
STALLTIME, verbose, fromatty, progress, quit_time, ttywidth
 1.63 27-Aug-2002  christos Add a -q <quit-time> flag to abort a transfer if it has stalled for <quit-time>
seconds. Ok'd by luke.
 1.62 26-Dec-2001  lukem branches: 1.62.2;
update copyrights
 1.61 23-Dec-2001  lukem Add -4 to force IPv4 and -6 to force IPv6 address usage.
From Hajimu UMEMOTO, via Mike Heffner of FreeBSD.

(FreeBSD has imported NetBSD's ftp as their ftp client;
Mike is sending back some of their local changes).
 1.60 19-Feb-2001  cgd convert to use getprogname()
 1.59 15-Nov-2000  lukem - implement "mreget"; as per "mget" but uses "reget" instead of "get"
- add -N netrc and $NETRC, as methods to select an alternative .netrc file
- cache local user name and home directory for further use
- in mget(), use docase() instead of a local version to do the case
conversion.
 1.58 01-Aug-2000  lukem - rename NO_QUAD to NO_LONG_LONG, QUAD* -> LL* and add ULL* (unsigned)
equivalents. name change suggested by Klaus Klein <kjk@netbsd.org>
- change defined(BSD4_4) || HAVE_SIN_LEN tests into HAVE_SOCKADDR_SA_LEN,
and set the latter if BSD4_4 exists
 1.57 30-Jul-2000  lukem clean up NO_QUAD support: create helper #defines and use as appropriate:
#define NOQUAD ! NOQUAD
------- ------ - ------
QUADF "%ld" "%lld"
QUADFP(x) "%" x "ld" "%" x "lld"
QUADT long long long
STRTOL(x,y,z) strtol(x,y,z) strtoll(x,y,z)
 1.56 28-Jul-2000  lukem no trailing , on last item in enum
 1.55 18-Jul-2000  lukem add support for FEAT and OPTS commands with `features' and `opts'.
(from RFC 2389).

add support for MLST & MLSD (machine parseble listings) with 'mlst', 'mlsd'
and 'pmlsd' (mlsd |$PAGER) commands. (from draft-ietf-ftpext-mlst-11)

rename remotesyst() to getremoteinfo(), and modify to parse the result from
FEAT (if supported), and take into account the support for the various
extensions such as MDTM, SIZE, REST (STREAM), MLSD, and FEAT/OPTS.
put each feature into one of the following categories:
- known to work (explicit FEAT)
- unknown but assume works until explicit failure, when it's
then tagged as `known not to work'.
- known not to work (FEAT succeeded but didn't return anything,
or was unknown and then explicit failure)
assign results into features[] matrix.

add support to getreply() so that an optional callback will be called
for each line received from the server except for the first and last.
this is used in FEAT (and MLST) parsing.

modify various commands to check if REST (STREAM), MDTM and SIZE are
explicitly or implicitly supported before using.

fix `syst' when verbose is off.

minor knf (indent goto labels by one space, etc).

simply various command usage handlers by assuming that argv != NULL except
for quit() and disconnect().
 1.54 01-May-2000  lukem convert to ANSI KNF
 1.53 01-May-2000  lukem * Add support for `fget localfile', which reads a list of filenames to
retrieve from localfile. Based on work by Darren Reed.
* Crank version.
* Update copyright dates.
 1.52 31-Jan-2000  lukem define private type `sigfunc' as
typedef void (*sigfunc) __P((int));
and replace use of sig_t and void (*)(int).

certain other OSes define sig_t differently to that (they add extra arguments),
and it causes problems due to function mismatches, etc...
 1.51 05-Dec-1999  lukem move version into separate file to reduce recompilation after version crank.
 1.50 05-Dec-1999  lukem crank version
 1.49 26-Nov-1999  lukem bump version
 1.48 12-Nov-1999  lukem - implement `set rprompt'; right side version of `set prompt'. depends on
EL_RPROMPT support i added to editline(3).
- allow $FTPPROMPT and $FTPRPROMPT to override defaults for the relevant
prompts
- move `%' formatting code from prompt() to expandbuf().
- implement `%.' and `%c', similar to the same % codes in tcsh(1)
(functionality I added to tcsh nearly 6 years ago), except that `%.'
always does `...trailing' and `%c' always does `/<x>trailing'.
- unknown `%foo' codes get printed as `%foo'
 1.47 11-Nov-1999  lukem - implement updateremotepwd(); update the global variable `remotepwd'
to contain the remote working directory.
- add `set prompt', a user configurable prompt. (defaults to `ftp> ').
the following escape characters a la tcsh(1) are supported: %/, %m,
%M, and %n.
- add global var `username'; used by prompt code
- fix a couple of minor memory leaks
- bump version
 1.46 09-Nov-1999  lukem - split the version string into product and version
- be consistent about reporting the version between:
+ status command
+ about:version URL fetch
+ User-agent sent in http requests
 1.45 03-Nov-1999  lukem support `about:version'. also display the version in the output of `status'.
 1.44 24-Oct-1999  lukem new features:
- add `usage'; displays the usage of a command.
implemented by calling the c_handler() with argc = 0, argv = "funcname".
- add `passive auto'; does the same as $FTPMODE=auto.
- add `set [option value]'; display all options, or set an option to a value.
- add `unset option'; unset an option.
- add getoptionvalue() to retrieve an option's value, and replace a few
global variables with calls to this.
- implement cleanuppeer(), which resets various bits of state back to
`disconnected'. call in disconnect() and lostpeer().
- support completing on `options'.
- improve recovery after a SIGINT may have closed the connection.
XXX: there's still a couple to fix

other stuff:
- various consistency fixes in the man page.
- ensure that the command usage strings in the code and man page match reality.
- mput/mget: check that the connection still exists before each xfer.
- minor cosmetic changes in confirm().
- set code correctly in sizecmd() and modtime()
- don't need \n in err() strings.
- change lostpeer to take an argument (rather than casting (sig_t)lostpeer
in signal handlers)
- knf and whitespace police.
 1.43 10-Oct-1999  lukem use sigjmp_buf for sigsetjmp(), instead of jmp_buf.
noted by Havard.Eidnes@runit.sintef.no.
 1.42 05-Oct-1999  lukem add TNFi copyright to all files i've done more than a minor amount of work to...
 1.41 05-Oct-1999  lukem enhancments from Marc Horowitz <marc@mit.edu> to improve connection timeouts:
* implement xsignal_restart(), which only sets the SA_RESTART flag if
specifically requested
* xsignal() is now a wrapper to xsignal_restart(). INFO, USR1, USR2 and WINCH
are restartable, ALRM, INT, PIPE and QUIT are not.
* improve getreply()'s timeout code to take advantage of the above.

other changes:
* improve wording of how globbing works for `classic' URLs (host:path).
suggested by John Refling <johnr@imageworks.com> in relation to PRs
[bin/8519] and [bin/8520]
* always compile in the `edit' command even if NO_EDITCOMPLETE defined.
it's just a no-op in the latter case, which is more consistent to
the users.
* always compile in about: support (i.e, remove NO_ABOUT).
i'm entitled to some vanity in this program...
* clean up some whitespace
 1.40 01-Oct-1999  lukem If EPSV or EPRT fails, disable epsv4 for the rest of the current connection.
the disabled state can be overridden by toggling epsv4.

(I got sick of the errors about EPSV not being supported on almost
every server I connect to. This way we retain support for epsv4, but
it's not so whiny after the first failure...)
 1.39 01-Oct-1999  lukem prefix the global variables in ftp_var.h with GLOBAL, which defaults
to "extern" if it's not set. define GLOBAL to (empty) in main.c.
this effectively moves all the globals into main.c whilst retaining
namespace access to them in other source files.
(global vars in header files confuse foreign linkers)
 1.38 01-Oct-1999  lukem * don't bother with "pathnames.h"; it contained just a single #define.
instead, include <paths.h> in the files that need it
* slightly reorg ftp_var.h
 1.37 30-Sep-1999  lukem * In the !NI_NUMERICHOST case (i.e, getaddrinfo() challenged systems), portnum
should be in host order. found/fixed by Matthias Pfaller <leo@dachau.marco.de>
* parse_url(): improve checking of portnum, and add an extra argument to pass
back the parsed portnum to the caller (reduces a bit of code duplication)
* Move the KAME/WIDE copyrights after the BSD/TNFi ones. Since there was
significantly less code added under the former, it's only fair on the latter.
 1.36 22-Sep-1999  lukem branches: 1.36.2;
* add support for `xferbuf', which sets both `sndbuf' and `rcvbuf'
* document the above three commands
* rototill the way the sndbuf and rcvbuf work. remove resetsockbufsize()
* use the appropriate socket buffer size as the size of the buffer that
the read()/write() loops use. speeds up things in some cases.
 1.35 11-Jul-1999  itojun add epsv4 command, which enable/disable the use of EPSV/EPRT.

this is mainly for (hypothetical) ftp server which disconnect clients
that use EPSV/EPRT. I've never seen any ftp server like this, but
epsv4 command may be of use when such an ftp server is found.
 1.34 02-Jul-1999  itojun add dual-stack (IPv4/v6) support. hope I broke no other part...
 1.33 29-Jun-1999  lukem [fear this; more ftp hacking from lukem :-]

features:
 1.32 20-Jun-1999  cgd split the "SMALL" #ifdefs into ones for NO_ABOUT, NO_EDITCOMPLETE, and
NO_PROGRESS. -DSMALL still implies all of those. progress meter support
isn't necessary for the smallest possible ftp client, but it adds very
little space and makes users' lives much better. Therefore, it should
be enabled for installation media if at all possible.
 1.31 22-Mar-1999  lukem branches: 1.31.2;
* implement -R; restart non-proxied command-line FTP xfers
* fix fetch_ftp() so that hcode parsing is not done for file:// urls
(a } in the wrong place, and code at the wrong indent level...)
* change outfile to being a global (so it gets correctly reset)
* change parse_url to not remove leading '/' for non ftp urls.
whilst this is not totally rfc1738 compliant, other code kinda
assumes this is the case, and it doesn't hurt
 1.30 08-Mar-1999  lukem * add url_decode() - `in-place' decode %xx escapes in a given url component
* parse_url()
- only look for user[:pass] for an ftp url (per rfc1738)
- strip leading /'s in an ftp url. (almost per rfc 1738)
* fetch_url()
- decode a copy of the path and use that to build local filenames
- send port in http Host: header (suggested by cgd@netbsd.org)
* fetch_ftp()
- url_decode() the user, pass and path
- fix splitting of path into dir & file (partially from [bin/7073])
- don't bother caching the last host; it can cause problems when
using ftp:// transfers, or when the user changes between xfers
* improve documentation of auto-fetched url arguments (especially regarding
escape sequences in ftp:// urls)
* some whitespace & copyright updates

this should fix [bin/7073] William O Ferry <woferry@warp.wofme.com>,
as well as the metaissues raised in that PR.
 1.29 23-Jan-1999  lukem support -f, which forces a cache flush for http xfers using either
`Pragma: no-cache' (http/1.0) or `Cache-Control: no-cache' (http/1.1)
 1.28 27-Dec-1998  lukem be saner about the method to determine which $proxy to use
 1.27 25-Nov-1998  christos more is not less on the systems that don't have less. Make "more" the default
pager if $PAGER is not set in the name of portability.
 1.26 08-Aug-1998  lukem pull in <netinet/in.h> and <arpa/inet.h> for in_{addr,port}_t (more portable)
 1.25 10-Jul-1998  thorpej Add "sndbuf" and "rcvbuf" commands for setting the socket buffer sizes,
which in turn can allow the use of larger TCP windows. This is a work in
progress; there is not yet support for specifying global defaults or
user prefrences on a host/network basis.
 1.24 04-Jun-1998  lukem some fixes & enhancements from openbsd's ftp, with extra fixes by me:
* default to passive with active fallback. $FTPMODE modifies this behaviour.
-A forces active connection.
* support '-o outfile' for auto-fetched files. outfile can be a file,
`-' (for stdout), or '|command' (to output each file through command).
* support '-r waittime', which retries the connection after waittime seconds
if it fails.
* fix 'page file' when restart is non-zero.
* try all ip-addresses of a host in a http fetch (as the normal ftp fetch
does).

XXX: a ``broken pipe'' error sometimes occurs with -o '|command';
i haven't tracked this down yet.
 1.23 18-Jan-1998  lukem define MAX_IN_PORT_T and use instead of USHRT_MAX. suggested by cgd@netbsd.org
 1.22 18-Jan-1998  lukem * ensure buffer for username is initialised, so ^D on username prompt
doesn't use garbage for the username. from "Soren S. Jorvang" <soren@t.dk>
in [bin/4559]
* use in_port_t for ports, and USHRT_MAX instead of 0xffff
(from millert@openbsd.org)
* use `NULL' instead of `(.... *)0' where appropriate.
 1.21 01-Nov-1997  lukem * in recvrequest(), ignore restart_point unless "RETR"ieving. fixes problems
where a remote completion or `mget' would confuse the client a `restart'
had been issued beforehand. now, `restart' is remembered until an operation
that can actually use it is invoked.
* in sendrequest(), don't reset restart_point upon entry. fixes `restart'
for `put' operations.
* if `restart' is invoked with no arguments, print current setting instead
of displaying a usage
* consistently use printf("%qd", (long long)restart_point) when displaying
restart_point
* use strto[lq]() instead of atol() when parsing `mark' and `restart' values
* remove unnecessary strlen()s when result of previous snprintf() will do
* replace a few malloc()/strcpy()s with strdup()s
* use SECSPERHOUR instead of '3600'
 1.20 13-Sep-1997  lukem branches: 1.20.2;
Fixes from Todd Miller <Todd.Miller@courtesan.com>:
* use size_t instead of int in places
* use symbolic constants when using access()
 1.19 23-Aug-1997  lukem support $TMPDIR
 1.18 18-Aug-1997  lukem bugs fixed:
* don't interpret '-' or '|' when a local filename is determined from
the remote name (i.e, in mget, and in get with only one argument).
This is implemented using an extra argument to recvrequest().
Fixes a major security hole.
* clean up memory leak when using globulize()
* clean up a couple of comments
* fix wording in TNF copyright

features added:
* support for TIS fwtk gate-ftp servers:
* read defaults from $FTPSERVER && $FTPSERVERPORT
* start in gate-ftp mode if invoked as 'gate-ftp'
* toggle or set with 'gate [host [port]]'
 1.17 20-Jul-1997  lukem * use RCSID() && COPYRIGHT() macros
* cleanup for WARNS=1 (including some ugly '(void)&var' bits wrapped in
#ifdef __GNUC__ to shut up gcc warnings WRT setjmp/longjmp)
* use strtol() instead of atoi(), and more extensively check result of
conversion
* use u_int16_t instead of short or int for TCP port addresses
 1.16 14-Apr-1997  lukem More enhancements/bugfixes (when will it end?)
* differentiate between being connected, and being logged in
* cleanup some text messages
* support username & password ftp URLs (ftp://user:pass@host/) in non-proxy
situations; assume proxy supports it for proxy situations.
* cd to / before performing any autofetch transfers
* use strncasecmp in URL parsing. fix from <Todd.Miller@courtesan.com>
 1.15 14-Mar-1997  christos SMALLFTP->SMALL; remove SMALL from Makefile
 1.14 13-Mar-1997  lukem Features:
* support remglobbing of auto_fetch arguments
* new flag - '-e'; disable editing
* "page file" == "get file |${PAGER-less}"

Bugfixes/cleanup:
* consistently use a trailing '.' on messages
* code cleanup, including buffer overrun fixes, use puts
and putchar in places, etc (inspired by OpenBSD mods)
* disable progress bar when local-file is a pipe or '-'
* skip \r in http headers
* fix remote ftpd slash bug more elegantly (so it works with ////)
* abort_remote(): check if cout==NULL before using it. should fix [bin/3273]
* fixed up cosmetic problems when complete_remote() generated errors from the
remote server (such as "no files found", "login with user and pass", ...)
done by adding extra argument to remglob(), which is a pointer to an error
buffer to put messages in rather than printing to stdout.
 1.13 01-Feb-1997  lukem [Yet Another Huge Ftp Commit - hopefully the last for a while,
barring any more little things people want added ...]

New features:
* progressmeter is now asynchronous, so "stalled" transfers can be
detected. "- stalled -" is displayed instead of the ETA in this case.
When the xfer resumes, the time that the xfer was stalled for is
factored out of the ETA. It is debatable whether this is better than
not factoring it out, but I like it this way (I.e, if it stalls for 8
seconds and the ETA was 30 seconds, when it resumes the ETA will still
be 30 seconds).
* verbosity can be disabled on the command line (-V), so that in auto-fetch
mode the only lines displayed will be a description of the file, and
the progress bar (if possible)
* if the screen is resized (and detected via the SIGWINCH signal), the
progress bar will rescale automatically.

Bugs fixed:
* progress bar will not use the last character on the line, as this can
cause problems on some terminals
* screen dimensions (via ioctl(TIOCWINSZ)) should use stdout not stdin
* progressmeter() used some vars before initialising them
* ^D will quit now. [fixes bin/3162]
* use hstrerror() to generate error message for host name lookup failure.
* use getcwd instead of getwd (it should have been OK, but why tempt fate?)
* auto-fetch transfers will always return a positive exit value upon failure
or interruption, relative to the file's position in argv[].
* remote completion of / will work, without putting a leading "///".
This is actually a bug in ftpd(1), where "NLST /" prefixes all names
with "//", but fixing every ftpd(1) is not an option...
 1.12 19-Jan-1997  lukem New features:
* Command line editing via editline(3) library.
* Context sensitive command and file completion, including remote files.

Enhancements to auto-fetch feature:
* Support for http:// URLs using the http protocol, including proxy HTTP
support via $htty_proxy if it's defined.
* The connection is kept open between successive files on the same host.
(obviously, this does not count for http requests.)
* Return value of ftp is 0 on no error, or the offset in argv[] of the file
which failed (i.e., argv[x] failed, ftp returns x).
* If the path in an ftp URL or classic format line has a trailing '/',
cd to the path and enter interactive mode. Fixes [bin/3011], albiet
requiring the user to help ftp in determining the operation.

Other changes:
* '-P port' works for normal ftp, and is the default for all classic style
auto-fetch transfers and for ftp URLs that don't specify the port.
(previously it would just work for the first xfer.)
* Some code moved into separated files along logical divisions.
* Editing and completion can be compiled out with -DSMALLFTP.
 1.11 09-Jan-1997  tls RCS ID police
 1.10 06-Dec-1996  lukem functionality mods:
* implement 'progress bar/meter' (inspired by ncftp). use 'progress' to
toggle on. it will display current file size to 5 digits, automatically
determining suffix (up to 16384 P (petabytes) == 2^64).
* 'ls' now uses NLST (unadorned listing), a la older ftp clients. 'dir'
still does LIST (long listing). idea from John Nemeth <jnemeth@cue.bc.ca>

bug fixes:
* return first line of reply in reply_string[] from getreply(), instead
of last line. This fixes [bin/741] (parsing of SYST), and also means
that SIZE and MDTM messages will be parsed correctly if they're longer
than 1 line.
* parse URL-style auto-ftps that have no filename correctly
(e.g, ftp://host, ftp://host/, ftp://host/dir/). pointed out by
Jaromir Dolecek <dolecek@saruman.ics.muni.cz>
* pass the correct size array in 2nd arg of utimes() when setting the
modification time
 1.9 28-Nov-1996  lukem More features, some of which were inspired by changes that
friedman@gnu.ai.mit.edu (Noah Friedman) made to his modified ftp:
- implement "lpwd" - local pwd
- implement "preserve" - toggle preserving of file modification
times on retrieved files
- allow for explicit "on" or "off" arg to toggle commands
- "exit" synonym for "quit", "msend" synonym for "mput"
- in confirmation mode, allow 'a' (yes to rest of current command),
and 'p' (turn off prompt mode, as if 'prompt off' was done,
effective immediately)
- "modtime" returns time formatted as localtime, not GMT

Bug fixes:
- check for extraneous args on commands
- cleanup const usage, line formatting
- create 0 length temporary file in remglob() to prevent symlink games
(from OpenBSD)
- check length of filename of ~/.netrc (from OpenBSD)
 1.8 25-Nov-1996  lukem cleanups/bugfixes:
- don't echo 'ACCT' parameter when debugging (a la 'PASS')
- Fix checking of directory access for "/foo", the parent
directory is "/", not "" (from FreeBSD)
- remove trailing whitespace on lines
- add any missing NetBSD tags
- cleanups to man page, includinging sorting options description

feature additions:
- variable sized hash marks (from [bin/683], but done in the hash command
as an optional arg)
- more user-friendly transfer time printing (from FreeBSD, with mods)
- '-p' command line option to jump into PASV mode (closes [bin/2857],
but with an option rather than checking argv[0])
- SIGINFO support for printing xfer stats when sending/receiving requests
- '-P port' for changing the port to connect to (from thorpej@netbsd.org)
- '-a': bypass normal login, and try anonymous login (from OpenBSD
via thorpej)
- autofetch files via url (ftp://...) or "classic" (host:/file)
(from OpenBSD via thorpej)
- 'ftp' synonymous with 'open' (from FreeBSD)
 1.7 15-Sep-1995  pk Avoid falling off the end of a fixed-length array (from John F. Woods; PR#1431).
 1.6 08-Sep-1995  tls Sync with 4.4lite2
 1.5 29-Aug-1994  mycroft branches: 1.5.2;
Add RCS ids.
 1.4 25-Aug-1994  cgd passive mode support, from David Carrel <carrel@cisco.com>, and cleaned
up for the new ftp sources by me.
 1.3 25-Aug-1994  cgd clean up import, no local changes.
 1.2 01-Aug-1993  mycroft Add RCS identifiers.
 1.1 21-Mar-1993  cgd branches: 1.1.1; 1.1.2;
Initial revision
 1.1.2.1 25-Aug-1994  cgd import from 4.4-Lite
 1.1.1.2 05-Sep-1995  tls imported from 44lite2
 1.1.1.1 21-Mar-1993  cgd initial import of 386bsd-0.1 sources
 1.5.2.2 29-Aug-1994  mycroft Add RCS ids.
 1.5.2.1 29-Aug-1994  mycroft file ftp_var.h was added on branch netbsd-1-0 on 1994-08-29 03:09:18 +0000
 1.20.2.2 10-Nov-1998  cgd pull up rev(s) 1.22-1.26 from trunk. (feyrer)
 1.20.2.1 18-Nov-1997  mellon Pull rev 1.21 up from trunk (lukem)
 1.31.2.1 25-Jun-1999  cgd pull up rev 1.32 from trunk. (cgd)
 1.36.2.1 27-Dec-1999  wrstuden Pull up to last week's -current.
 1.62.2.1 10-Feb-2003  jmc Pullup rev 1.64 (requested by grant in ticket #1115)
Separate progress reporting into a utility set of routines. Use
these to provide a standalone progress binary and integrate support
for this into sysinst.
 1.69.2.3 24-Jul-2005  tron Pull up revision 1.72 (requested by lukem in ticket #606):
Add NO_USAGE and NO_DEBUG so that we can fit in the floppies again.
 1.69.2.2 09-May-2005  tron Pull up revision 1.71 (requested by lukem in ticket #266):
gratuitous whitespace cleanup (before someone else jumps the gun...)
 1.69.2.1 09-May-2005  tron Pull up revision 1.70 (requested by lukem in ticket #265):
Implement a timeout on the accept(2) in dataconn() and the
connect(2) in xconnect() by temporarily setting O_NONBLOCK
on the socket and using xpoll() to wait for the operation
to succeed.
The timeout used is the '-q quittime' argument (defaults to
60s for accept(2), and the system default for connect(2)).
Idea inspired by discussion with Chuck Cranor.
This may (indirectly) fix various problems with timeouts
in active mode through broken firewalls.
Implement xpoll() as a wrapper around poll(2), to make it
easier to replace on systems without a functional poll(2).
Unconditionally use xpoll() instead of conditionally using
select(2) or poll(2).
 1.75.6.2 22-Jul-2007  lukem Replace HAVE_SOCKADDR_SA_LEN with defined(HAVE_STRUCT_SOCKADDR_SA_LEN)
 1.75.6.1 22-Jul-2007  lukem file ftp_var.h was added on branch matt-mips64 on 2007-07-22 05:02:51 +0000
 1.75.4.1 09-Jan-2008  matt sync with HEAD
 1.77.6.1 18-May-2008  yamt sync with head.
 1.78.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.80.6.1 13-May-2009  jym Sync with HEAD.

Third (and last) commit. See http://mail-index.netbsd.org/source-changes/2009/05/13/msg221222.html
 1.81.12.1 25-Feb-2013  tls resync with head
 1.81.8.2 27-Aug-2016  bouyer Apply patch, requested by nonaka in ticket #1375:
src/usr.bin/ftp/cmds.c: patch
src/usr.bin/ftp/fetch.c: patch
src/usr.bin/ftp/ftp.1: patch
src/usr.bin/ftp/ftp.c: patch
src/usr.bin/ftp/ftp_var.h: patch
src/usr.bin/ftp/main.c: patch
src/usr.bin/ftp/ssl.c: patch
src/usr.bin/ftp/ssl.h: patch
src/usr.bin/ftp/version.h: patch
Update ftp(1) to version 20150912, adding https via proxy support.
 1.81.8.1 17-Dec-2013  bouyer Apply patch, requested by tron in ticket #997:
usr.bin/ftp/Makefile patch
usr.bin/ftp/cmds.c patch
usr.bin/ftp/cmdtab.c patch
usr.bin/ftp/extern.h patch
usr.bin/ftp/fetch.c patch
usr.bin/ftp/ftp.1 patch
usr.bin/ftp/ftp.c patch
usr.bin/ftp/ftp_var.h patch
usr.bin/ftp/main.c patch
usr.bin/ftp/progressbar.c patch
usr.bin/ftp/ssl.c patch
usr.bin/ftp/ssl.h patch
usr.bin/ftp/util.c patch
usr.bin/ftp/version.h patch

Add HTTPS support to ftp(1).
 1.81.6.1 23-Jan-2013  yamt sync with head
 1.82.8.2 13-Mar-2016  martin Pull up following revision(s) (requested by nonakap in ticket #1133):
usr.bin/ftp/fetch.c: revision 1.208-1.221
usr.bin/ftp/cmds.c: revision 1.136-1.137
usr.bin/ftp/ssl.c: revision 1.5
usr.bin/ftp/ftp.c: revision 1.165-1.166
usr.bin/ftp/ftp_var.h: revision 1.84

Workaround const issues of SSL_set_tlsext_host_name.

Use the proper format "[IPv6 address]:port" when reporting connection
attempts to IPv6 endpoints.

(Hopefully) fix build without IPv6 support

Try to factor out some code, this is completely out of control.

Separate no_proxy handling.

Factor the proxy handling code out.

Fix compile failure without WITH_SSL.

PR/50438: NONAKA Kimihiro: ftp(1): CONNECT method support

make DPRINTF/DWARN always statements.

Fix to connect https via proxy.

Fix ttyout message.

Simplify and factor out connect message

Split the position/size parsing into a separate function.

Mark function as only needed with ssl.

Fix downloads of local files using file:// URLs

Initialize the token match pointer.

use sizeof() and array notation.
CID 1354295: Array overrun.
 1.82.8.1 14-Apr-2015  snj Pull up following revision(s) (requested by christos in ticket #685):
usr.bin/ftp/ftp_var.h: revision 1.83
Increase the buffer limit; otherwise files in:
http://www.taxdetective.ca/Samples/sampledatafiles.html
fail.
 1.84.8.3 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1763:

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/cmds.c up to 1.141
usr.bin/ftp/complete.c up to 1.47
usr.bin/ftp/domacro.c up to 1.23
usr.bin/ftp/extern.h up to 1.82
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp.c up to 1.174
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/progressbar.c up to 1.24
usr.bin/ftp/progressbar.h up to 1.9
usr.bin/ftp/ssl.c up to 1.12
usr.bin/ftp/ssl.h up to 1.5
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

ftp(1): validate address from PASV and LPSV response.
ftp(1): use raw write(2) instead of fwrite(3) to avoid stream
corruption because of the progress bar interrupts.
Fixes for PR 56219 and PR 55857.
PR 57003: Support relative redirects.
 1.84.8.2 12-Sep-2022  martin Backout ticket #1763 for now - trust anchors are not solved.
 1.84.8.1 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1763:

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/cmds.c up to 1.141
usr.bin/ftp/complete.c up to 1.47
usr.bin/ftp/domacro.c up to 1.23
usr.bin/ftp/extern.h up to 1.82
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp.c up to 1.174
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/progressbar.c up to 1.24
usr.bin/ftp/progressbar.h up to 1.9
usr.bin/ftp/ssl.c up to 1.11
usr.bin/ftp/ssl.h up to 1.5
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

ftp(1): validate address from PASV and LPSV response.
ftp(1): use raw write(2) instead of fwrite(3) to avoid stream
corruption because of the progress bar interrupts.
Fixes for PR 56219 and PR 55857.
PR 57003: Support relative redirects.
 1.85.6.3 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1523

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/ssl.c up to 1.12
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

PR 57003: Support relative redirects.
 1.85.6.2 12-Sep-2022  martin Back out ticket #1523 for now - trust anchor validation is not solved.
 1.85.6.1 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1523

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/ssl.c up to 1.11
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

PR 57003: Support relative redirects.
 1.86.2.1 13-Oct-2024  martin Pull up following revision(s) (requested by riastradh in ticket #970):

tests/usr.bin/Makefile: revision 1.42
usr.bin/ftp/ruserpass.c: revision 1.34
usr.bin/ftp/main.c: revision 1.130
usr.bin/ftp/ssl.c: revision 1.17
usr.bin/ftp/main.c: revision 1.131
usr.bin/ftp/ssl.c: revision 1.18
usr.bin/ftp/main.c: revision 1.132
usr.bin/ftp/ssl.c: revision 1.19
usr.bin/ftp/main.c: revision 1.133
distrib/sets/lists/tests/mi: revision 1.1342
usr.bin/ftp/ftp.1: revision 1.151
usr.bin/ftp/ftp.1: revision 1.152
usr.bin/ftp/progressbar.c: revision 1.25
usr.bin/ftp/ftp.1: revision 1.153
usr.bin/ftp/progressbar.c: revision 1.26
usr.bin/ftp/ftp.1: revision 1.155
usr.bin/ftp/ftp.1: revision 1.156
usr.bin/ftp/fetch.c: revision 1.239
usr.bin/ftp/ftp.1: revision 1.157
usr.bin/ftp/ftp.1: revision 1.158
usr.bin/ftp/ftp.1: revision 1.159
usr.bin/ftp/ftp_var.h: revision 1.87
etc/mtree/NetBSD.dist.tests: revision 1.208
usr.bin/ftp/ftp_var.h: revision 1.88
usr.bin/ftp/ftp_var.h: revision 1.89
usr.bin/ftp/cmds.c: revision 1.142
usr.bin/ftp/util.c: revision 1.168
usr.bin/ftp/cmds.c: revision 1.143
tests/usr.bin/ftp/custom_headers.sh: revision 1.1
usr.bin/ftp/ssl.c: revision 1.20
usr.bin/ftp/complete.c: revision 1.48
tests/usr.bin/ftp/Makefile: revision 1.1
tests/usr.bin/ftp/t_custom_headers.sh: revision 1.1
usr.bin/ftp/fetch.c: revision 1.240
usr.bin/ftp/fetch.c: revision 1.241
usr.bin/ftp/ftp.c: revision 1.176
usr.bin/ftp/ftp.c: revision 1.177
(all via patch)

ftp(1): wording and formatting improvements

Fix grammar issue with "Support values" reported in private mail.
Document all file transfer types in "type" and cross-reference that.
Consistency fixes in describing file transfer parameters and types.

Fix some mandoc -Tlint issues (except "useless macro: Tn").

Add -b <buflen> to specify the buffer size.

ftp: bump FTPBUFLEN from 4kB to 16kB
sourceforge.net returns a 5kB content-security-policy.
Analyzed by mlelstv@ who reports usual limits are between 4kB and 48kB.
default is now 16K

ftp: improve -b documentation

Order -b bufsize in the synopsis.

Document the actual default value.

ftp: improve units used in comments and errors
Use "KiB" instead of "K" in errors.
Clarify related comments.

pass some lint.

PR/58581: Sunil Nimmagadda: Add flag to allow specifying extra http header
fields.

ftp(1): Nix trailing whitespace in man page.
No functional change intended.

PR bin/58581: ftp(1) should allow specifying header fields in http requests
fix markup (h -> H), explain about multiple headers, fix usage (from RVP)

Don't forget the dot, use the intended macro name (I think),
and improve the wording a little. (All related to the -H option.)
ftp(1): Add test for custom HTTP header fields.

Based on a patch from Sunil Nimmagadda.

PR bin/58581: ftp(1) should allow specifying header fields in http
requests
 1.88.2.1 02-Aug-2025  perseant Sync with HEAD
 1.3 25-Aug-1994  mycroft Clean up deleted files.
 1.2 01-Aug-1993  mycroft Add RCS identifiers.
 1.1 21-Mar-1993  cgd branches: 1.1.1;
Initial revision
 1.1.1.1 21-Mar-1993  cgd initial import of 386bsd-0.1 sources
 1.135 29-Nov-2024  lukem ftp: order getopt Upper before lower

Consistently order options in getopt and the switch
with the upper case option before the lower case option.
This makes it easier to cross-reference with -? and ftp(1).
No functional change.
 1.134 29-Nov-2024  lukem ftp: help improvements

Document -? as a separate mode.
Document -H HEADER in the usage.
Clarify units for -b and -x.
Consistent argument names between ftp -? and ftp(1).
 1.133 30-Sep-2024  christos fix markup (h -> H), explain about multiple headers, fix usage (from RVP)
 1.132 25-Sep-2024  christos PR/58581: Sunil Nimmagadda: Add flag to allow specifying extra http header
fields.
 1.131 25-Sep-2024  christos pass some lint.
 1.130 18-Feb-2024  christos branches: 1.130.2;
Add -b <buflen> to specify the buffer size.
 1.129 25-Feb-2023  mlelstv Add option sslnoverify to control validation of SSL certificates.
Add netrc processing to fetch-mode (URL on command line) to enable options and autologin
via netrc.
Fix SSL cleanup in some error paths.

Certificate validation is now enabled by default. Set FTPSSLNOVERIFY=1 in environment
or configure a corresponding init macro via netrc to not validate certs (required if
you haven't installed a required CA certificate for OpenSSL).

Discussed with lukem@ on icb.
 1.128 09-Oct-2021  lukem branches: 1.128.2;
ftp: fix -? more portably

Start the optstring with ":" (which implicitly disables opterr),
and handle '?' with optopt=='?' separately to ':'.
Fixes -? display of full usage on other platforms.
 1.127 18-Jul-2020  lukem ftp: add -? for help. improve synopsis

Add -? to display usage synopsis and help to stdout.
This allows for "ftp -? | less", which is more user friendly.
Errors still show usage to stderr.
Consistency improvements in some usage text.
 1.126 04-Feb-2019  mrg branches: 1.126.2;
- add justquit() that always exits. use it to avoid unreachable code.
 1.125 04-Mar-2018  dholland branches: 1.125.4;
Don't use the local username as the default anonftp password.

once upon a time doing this was part of the social glue that held the
community together, but that was a long time ago, and now it's just an
information leak.

proposed on tech-userlevel in 2008, then apparently forgotten :-|
 1.124 25-Nov-2017  christos Make outfile always allocated, free it to set it to NULL, and don't move it
around.
 1.123 23-Apr-2015  lukem branches: 1.123.8;
Add -x xferbufsize to set xferbuf size.

Implement -x xferbufsize set the socket send and receive buffer size,
as per 'xferbuf' in interactive mode.

Patch from Nicholas Mills (via private mail), with minor adjustment by me.
 1.122 22-Dec-2012  christos branches: 1.122.8;
document https support, mentioning certificates are not validated, and obey
https_proxy.
 1.121 21-Dec-2012  christos PR/47276: Add https support
 1.120 10-Dec-2011  lukem branches: 1.120.2; 1.120.6;
Move determination of socket buffer sizes from startup to the first
time a socket is used, as the previous logic assumed AF_INET sockets
were available (which they may not be in an IPv6-only system).
Per discussion with Maxim Konovalov and the FreeBSD problem 162661.
 1.119 16-Sep-2011  joerg branches: 1.119.2;
Use __dead
 1.118 14-Aug-2011  christos fix gcc-4.5 warnings
 1.117 13-Jul-2009  roy Rename internal getline() function to get_line() so it does
conflict with the soon to be added getline(3) libc function.
 1.116 24-Apr-2009  lukem Revert incorrect change made as part of the WARNS=4.
Fixes "ftp host [port]" on the CLI.
Noted by Geoff Wing.
 1.115 12-Apr-2009  lukem Fix numerous WARNS=4 issues (-Wcast-qual -Wsign-compare).
 1.114 12-Apr-2009  lukem fix -Wshadow issues
 1.113 09-Sep-2008  gmcgarry branches: 1.113.6;
getline() returns int. Don't cast to unsigned value and compare with -1.
 1.112 13-Aug-2008  lukem Use AF_INET instead of AF_UNSPEC as the default family if !defined(INET6).
Avoids problem reported by Matthias Scheler <tron@NetBSD.org>.
 1.111 21-Jul-2008  lukem Add TNF to __COPYRIGHT
 1.110 21-Jul-2008  lukem Remove the \n and tabs from the __COPYRIGHT() strings.
Tweak to use a consistent format.
 1.109 10-May-2008  skd Add epsv6 and epsv to disable extended passive mode for ipv6 or both ipv4 and ipv6 respectively. This hack is due to our friends a Juniper Networks who break
epsv in ipv6. Should be fixed in ScreenOS 6.2.X.
 1.108 05-May-2008  lukem branches: 1.108.2;
Only attempt to el_parse() a command unknown by the default parser
if editing is enabled.
Fixes PR bin/38589
 1.107 28-Apr-2008  martin Remove clause 3 and 4 from TNF licenses
 1.106 02-Dec-2007  wiz branches: 1.106.6;
Sort options.
 1.105 22-May-2007  lukem branches: 1.105.4;
* main: call tzset() to ensure TZ is setup for other <time.h> functions.
* remotemodtime(): use strptime() to parse the reply.
* fetch_url(): ensure struct tm is zeroed before calling strptime().
 1.104 17-Apr-2007  lukem * Implement -s srcaddr; uses srcaddr as the local IP address for all
connections.
Based on code in the version of ftp that FreeBSD had before they
replaced it with lukemftp.
* Move error message handling into ftp_connect() rather than in the
caller, so that more specific error reporting can occur.
* Improve consistency of various warning and error messages.
 1.103 12-Apr-2007  lukem Fix using /usr/bin/ftp as a slave process on the end of a pipe by calling
fflush(stdout) before each command.
Taken from FreeBSD's pre-lukemftp version of ftp.
 1.102 13-Dec-2006  christos gcc4 does not care about &foo; use volatile instead. From Anon Ymous
 1.101 31-Jan-2006  christos rename debug to ftp_debug. grr libssh.
 1.100 31-Jan-2006  christos Rename xfoo() to ftp_foo() to avoid collisions with libssh. Don't ask.
 1.99 29-Jun-2005  christos Add NO_USAGE and NO_DEBUG so that we can fit in the floppies again.
 1.98 10-Jun-2005  jmc Need to #ifdef around declaration of ch with NO_EDITCOMPLETE so this compiles
on rescue/miniroot's
 1.97 10-Jun-2005  lukem Implement:
int getline(FILE *stream, char *buf, size_t buflen, const char **errormsg)
Read a line from the FILE stream into buf/buflen using fgets(), so up
to buflen-1 chars will be read and the result will be NUL terminated.
If the line has a trailing newline it will be removed.
If the line is too long, excess characters will be read until
newline/EOF/error.
Various -ve return values indicate different errors, and errormsg
will be changed to an error description if it's not NULL.

Convert to use getline() instead of fgets() whenever reading user input
to ensure that an overly long input line doesn't leave excess characters
for the next input operation to accidentally use as input.

Zero out the password & account after we've finished with it.

Consistently use getpass(3) (i.e, character echo suppressed) when
reading the account data. For some reason, historically the "login"
code suppressed echo for Account: yet the "user" command did not!

Display the hostname in the "getaddrinfo failed" warning.

Appease some -Wcast-qual warnings. Fixing all of these requires
significant code refactoring. (mmm, legacy code).
 1.96 20-May-2005  jdc Make this compile on sparc64 (size_t != int).
 1.95 19-May-2005  lukem Use size_t instead of int where appropriate.
 1.94 13-May-2005  lukem * Correct the "optlen" argument passed to getsockopt(3) and setsockopt(3)
in various places. Fixes a problem noted by Allen Briggs.
* Improve warning printed when connect(2) for the data channel fails.
 1.93 11-May-2005  lukem Use socklen_t instead of int as the 5th argument to getsockopt().
Improve invocation of setsockopt() and associated failure messages.
 1.92 10-May-2005  lukem Prevent an overly-long input line causing a core dump when editing is enabled.
Issue noted by Ryoji Kanai in FreeBSD Problem Report # 77158.
 1.91 11-Apr-2005  lukem gratuitous whitespace cleanup (before someone else jumps the gun...)
 1.90 21-Jul-2004  lukem branches: 1.90.2;
Slightly rework SIGINT handling; if we're exiting the auto-fetch stuff
and sigint_raised is non-zero, reset the handler for SIGINT to SIG_DFL
and raise(SIGINT) so that the appropriate wait(3) status is setup.
Based on solution proposed by Ognyan Kulev.
This should really fix PR [pkg/26351].
 1.89 20-Jul-2004  lukem If an ftp auto-fetch transfer is interrupted by SIGINT (usually ^C),
exit with 130 instead of 1 (or rarely, 0).
This allows an ftp auto-fetch in a shell loop to correctly terminate the loop.
Should fix PR [pkg/26351], and possibly others.
 1.88 09-Jul-2004  wiz Add -q to usage. From Kouichirou Hiratsuka in PR 26199.
 1.87 14-May-2004  christos PR/25566: Anders Magnusson: ftp(1) do not like large TCP windows.
Limit it to 8M.
 1.86 07-Aug-2003  agc Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22365, verified by myself.
 1.85 26-Jul-2003  salo netbsd.org->NetBSD.org
 1.84 14-May-2003  wiz Uppercase URL.
 1.83 27-Aug-2002  christos Add a -q <quit-time> flag to abort a transfer if it has stalled for <quit-time>
seconds. Ok'd by luke.
 1.82 05-Jun-2002  lukem - use setprogname()
- only support -6 if INET6 is defined
 1.81 05-Jun-2002  lukem - when showing the final progress bar, replace "00:00 ETA" with the
elapsed time. (suggested by simonb)
- actually display transfer stats after a URL fetch. (bug introduced a
*long* time ago)
- update copyright & version
 1.80 18-Mar-2002  thorpej Deal with const'ification if el_parse().
 1.79 26-Dec-2001  lukem update copyrights
 1.78 23-Dec-2001  lukem Add -4 to force IPv4 and -6 to force IPv6 address usage.
From Hajimu UMEMOTO, via Mike Heffner of FreeBSD.

(FreeBSD has imported NetBSD's ftp as their ftp client;
Mike is sending back some of their local changes).
 1.77 19-Feb-2001  cgd convert to use getprogname()
 1.76 09-Jan-2001  jdolecek call setlocale() on startup
 1.75 15-Dec-2000  lukem invoke cmdtab.c_handler()s with argv[0] == c_name instead of the
supplied name. that way the full (unambiguous) name is displayed in
error messages and usage strings.
 1.74 15-Nov-2000  lukem - implement "mreget"; as per "mget" but uses "reget" instead of "get"
- add -N netrc and $NETRC, as methods to select an alternative .netrc file
- cache local user name and home directory for further use
- in mget(), use docase() instead of a local version to do the case
conversion.
 1.73 18-Jul-2000  lukem add support for FEAT and OPTS commands with `features' and `opts'.
(from RFC 2389).

add support for MLST & MLSD (machine parseble listings) with 'mlst', 'mlsd'
and 'pmlsd' (mlsd |$PAGER) commands. (from draft-ietf-ftpext-mlst-11)

rename remotesyst() to getremoteinfo(), and modify to parse the result from
FEAT (if supported), and take into account the support for the various
extensions such as MDTM, SIZE, REST (STREAM), MLSD, and FEAT/OPTS.
put each feature into one of the following categories:
- known to work (explicit FEAT)
- unknown but assume works until explicit failure, when it's
then tagged as `known not to work'.
- known not to work (FEAT succeeded but didn't return anything,
or was unknown and then explicit failure)
assign results into features[] matrix.

add support to getreply() so that an optional callback will be called
for each line received from the server except for the first and last.
this is used in FEAT (and MLST) parsing.

modify various commands to check if REST (STREAM), MDTM and SIZE are
explicitly or implicitly supported before using.

fix `syst' when verbose is off.

minor knf (indent goto labels by one space, etc).

simply various command usage handlers by assuming that argv != NULL except
for quit() and disconnect().
 1.72 11-Jun-2000  lukem portability fixes for lukemftp:
* initconn(): use in_addr_t instead of u_int32_t when manipulating IPv6
addresses (and assume anything with ipv6 has in_addr_t; if not, i'll
add an autoconf test for it)
* ai_unmapped(): not all systems have sin_len; so only set #ifdef BSD4_4
* fix some lint
 1.71 31-May-2000  lukem Add support for 'ftp -u url file ...', to upload a list of files to given url.
Mostly based on [bin/10019] by Scott Aaron Bamford <sab@ansic.net>
 1.70 01-May-2000  lukem branches: 1.70.2;
convert to ANSI KNF
 1.69 28-Nov-1999  lukem * implement xsl_init() and xsl_add(); error checking forms of sl_{init,add}()
* fix bug where the second press of <TAB> on an empty word (i.e, list
all options) may have resulted in an strncmp() against NULL. (detected
by _DIAGASSERT())
 1.68 26-Nov-1999  lukem * complete_remote(): use remglob("", ...) instead of remglob(".", ...),
for listings of the current working directory; some ftp servers don't
like `NLST .'.
[noted by Giles Lean <giles@nemeton.com.au>]
* recvrequest(): treat remote=="" as remote==NULL when calling command().
(to support the above change)
* support `[user@]' in `[user@]host' and `[user@]host[:][path]'.
[based on idea (and initial code) from David Maxwell <david@fundy.ca>]
* `idle' may be invoked without any args
* reformat some comments
* reformat usage string in program and man page
* call updateremotepwd() after successful login, not after successful connect
* always call setsockopt(, IPPROTO_IP, IP_TOS, ) (et al); using #if
defined(IPPROTO_IP) doesn't work on certain foreign systems where
enums instead of #defines are used...
[noted by Matthias Pfaller <leo@dachau.marco.de>]
 1.67 12-Nov-1999  lukem - implement `set rprompt'; right side version of `set prompt'. depends on
EL_RPROMPT support i added to editline(3).
- allow $FTPPROMPT and $FTPRPROMPT to override defaults for the relevant
prompts
- move `%' formatting code from prompt() to expandbuf().
- implement `%.' and `%c', similar to the same % codes in tcsh(1)
(functionality I added to tcsh nearly 6 years ago), except that `%.'
always does `...trailing' and `%c' always does `/<x>trailing'.
- unknown `%foo' codes get printed as `%foo'
 1.66 11-Nov-1999  lukem - implement updateremotepwd(); update the global variable `remotepwd'
to contain the remote working directory.
- add `set prompt', a user configurable prompt. (defaults to `ftp> ').
the following escape characters a la tcsh(1) are supported: %/, %m,
%M, and %n.
- add global var `username'; used by prompt code
- fix a couple of minor memory leaks
- bump version
 1.65 24-Oct-1999  lukem new features:
- add `usage'; displays the usage of a command.
implemented by calling the c_handler() with argc = 0, argv = "funcname".
- add `passive auto'; does the same as $FTPMODE=auto.
- add `set [option value]'; display all options, or set an option to a value.
- add `unset option'; unset an option.
- add getoptionvalue() to retrieve an option's value, and replace a few
global variables with calls to this.
- implement cleanuppeer(), which resets various bits of state back to
`disconnected'. call in disconnect() and lostpeer().
- support completing on `options'.
- improve recovery after a SIGINT may have closed the connection.
XXX: there's still a couple to fix

other stuff:
- various consistency fixes in the man page.
- ensure that the command usage strings in the code and man page match reality.
- mput/mget: check that the connection still exists before each xfer.
- minor cosmetic changes in confirm().
- set code correctly in sizecmd() and modtime()
- don't need \n in err() strings.
- change lostpeer to take an argument (rather than casting (sig_t)lostpeer
in signal handlers)
- knf and whitespace police.
 1.64 13-Oct-1999  lukem * fix up confirm() (broke `a' and `p' in last commit)
* simplify main loop (don't need `top' variable any more)
* use a struct sockaddr_in6.sin6_addr for the result from inet_pton(),
rather than u_char buf[16]
* add a few more comments
 1.63 12-Oct-1999  lukem a few user interface and cosmetic tweaks:
* confirm(): move from util.c to cmds.c. display mnemonic string in its prompt.
add support for `q' (terminate current xfer), `?' (show help list)
* in various signal handlers, output a linefeed only if fromatty.
* if fgets(stdin) returned NULL (i.e, EOF), clearerr(stdin) because you don't
want future fgets to fail. this is not done for the fgets() in the main
command loop, since ftp will quit at that point.
* unless ftp is invoked with -a, don't retain the anonftp setting between
hosts (`ftp somehost:' sets anonftp, but you don't want that to `stick'
if you close that connection and open a new one).
 1.62 11-Oct-1999  lukem if editing is enabled when intr() is called, print a newline. (cosmetic fix)
 1.61 09-Oct-1999  lukem * use sigsetjmp()/siglongjump() instead of setjmp()/longjmp(); the latter
don't save the signal mask on some foreign systems.
* ensure signal handlers don't use stdio and do reset errno if they
don't exit with siglongjmp()
* use a common SIGINT handler for {send,recv}request()
 1.60 05-Oct-1999  lukem * factor out SIGINFO setting into a handler that is always active (but only
prints out info if bytes > 0). only set the handler if SIGINFO is defined
* hijack SIGQUIT to be the same as SIGINFO (foreign ports have this, and it's
annoying to have SIGQUIT dump core on netbsd when it prints info on other
systems)
* in {recv,send}request(), factor a lot of duplicated code out into a
`cleanup' section at the end
* rework shell() a bit
 1.59 05-Oct-1999  lukem add TNFi copyright to all files i've done more than a minor amount of work to...
 1.58 01-Oct-1999  lukem If EPSV or EPRT fails, disable epsv4 for the rest of the current connection.
the disabled state can be overridden by toggling epsv4.

(I got sick of the errors about EPSV not being supported on almost
every server I connect to. This way we retain support for epsv4, but
it's not so whiny after the first failure...)
 1.57 01-Oct-1999  lukem prefix the global variables in ftp_var.h with GLOBAL, which defaults
to "extern" if it's not set. define GLOBAL to (empty) in main.c.
this effectively moves all the globals into main.c whilst retaining
namespace access to them in other source files.
(global vars in header files confuse foreign linkers)
 1.56 01-Oct-1999  lukem * don't bother with "pathnames.h"; it contained just a single #define.
instead, include <paths.h> in the files that need it
* slightly reorg ftp_var.h
 1.55 30-Sep-1999  lukem * In the !NI_NUMERICHOST case (i.e, getaddrinfo() challenged systems), portnum
should be in host order. found/fixed by Matthias Pfaller <leo@dachau.marco.de>
* parse_url(): improve checking of portnum, and add an extra argument to pass
back the parsed portnum to the caller (reduces a bit of code duplication)
* Move the KAME/WIDE copyrights after the BSD/TNFi ones. Since there was
significantly less code added under the former, it's only fair on the latter.
 1.54 30-Sep-1999  lukem * fix initialisation of home[]
* fetch_url(): if path would be NULL, return strdup("")
 1.53 28-Sep-1999  lukem * add new commands:
lpage page local files
pdir as dir, but through your $PAGER
pls as ls, but through your $PAGER
* implement docase() (a la dotrans() et al) and use appropriately, rototilling
some duplicated code
* globulize(): modify to return a pointer to the strdup()ed result in all cases,
and hack the code that calls it to take this into account
* replace strcpy() and strncpy() with strlcpy()
* put(), getit(): use some aptly named local vars instead of argv[...]
* delint
 1.52 27-Sep-1999  lukem * idle(): rename to idlecmd(). certain linux distributions have an incompatible
prototype for idle() in <unistd.h> (which i thought was against namespace
and sensibility guidelines, but...)
* consistently use xsignal() instead of signal(). we get known behaviour
in all cases (SA_RESTART), which is good for some borken foreign systems.
* remove signal.h from most files; it's unnecessary now
* fetch_url(): use `long chunksize' instead of ssize_t; it's more portable, and
we're setting chunksize with strtol() anyway
* xsignal(): only use SA_RESTART if it exists. SunOS 4.x doesn't have it
but has the inverse (SA_INTERRUPT). the original function i was inspired
from had this support (lib/signal.c, W. Richard Stevens' `UNP 2nd ed Vol 1').
* remove <termios.h> from util.c; it should be unnecessary now
 1.51 24-Sep-1999  lukem if getsockopt() returns a buffer size of <= 0, set them to 8192.
this works around a problem in certain OS from Sun with a version < 5.6...
 1.50 22-Sep-1999  lukem branches: 1.50.2;
* add support for `xferbuf', which sets both `sndbuf' and `rcvbuf'
* document the above three commands
* rototill the way the sndbuf and rcvbuf work. remove resetsockbufsize()
* use the appropriate socket buffer size as the size of the buffer that
the read()/write() loops use. speeds up things in some cases.
 1.49 21-Sep-1999  lukem only default to epsv4 if INET6 is defined
 1.48 12-Jul-1999  lukem * change fetch_ftp() to be fully rfc 1738 compliant; if the URL contains
the an empty directory (e.g, between `some' and `path' in
`ftp://host/some//path'), then execute `CWD ' (without a path).
This command will probably fail on rfc 959 compliant servers, so
issue a warning in this case and bail. [noted by cgd].
(i wonder if the people who wrote rfc 1738 actually realised that this
requirement appears to contravene the spec for `cwd' in rfc 959 ?)
* replace isurl() with isipv6addr(), and use appropriately. fixes
auto-login with `classic ftp URLs' (e.g, `ftp somehost:')
* cleanup and rework some of the ipv6 stuff in parse_url()
* prevent potential coredump in fetch_ftp() when parsing `;type=X'
* KNF a few lines
* fix a couple of comments
* cleanup the man page a bit
 1.47 11-Jul-1999  itojun add epsv4 command, which enable/disable the use of EPSV/EPRT.

this is mainly for (hypothetical) ftp server which disconnect clients
that use EPSV/EPRT. I've never seen any ftp server like this, but
epsv4 command may be of use when such an ftp server is found.
 1.46 10-Jul-1999  christos isurl does not return a pointer!
 1.45 02-Jul-1999  itojun add dual-stack (IPv4/v6) support. hope I broke no other part...
 1.44 29-Jun-1999  lukem [fear this; more ftp hacking from lukem :-]

features:
 1.43 20-Jun-1999  cgd split the "SMALL" #ifdefs into ones for NO_ABOUT, NO_EDITCOMPLETE, and
NO_PROGRESS. -DSMALL still implies all of those. progress meter support
isn't necessary for the smallest possible ftp client, but it adds very
little space and makes users' lives much better. Therefore, it should
be enabled for installation media if at all possible.
 1.42 02-Jun-1999  lukem * fix gate mode to login as `user@realhost' rather than using PASSERVE;
the latter only seemed to work for TIS Gauntlet and not TIS fwtk.
thanks to simonb@netbsd.org for testing this. fixes [bin/5556].
* if EOF (e.g, ^D) is entered at a username/password/account prompt which
happens to use fgets(), exit the login rather than treating EOF as CR.
* don't use the comma operator where separate statements are valid
* always use snprintf to copy stuff into malloced buffers, just in case
typos creep in and mean that the buffer ends up being overflowed
 1.41 17-May-1999  lukem only enable editing by default if stdin is a tty.
noticed by Onno van der Linden <onno@simplex.nl>
 1.40 31-Mar-1999  lukem branches: 1.40.2;
when using editline, work on the correct buffer. fixes problem where blank
input lines were being added to the history buffer.
 1.39 22-Mar-1999  lukem * implement -R; restart non-proxied command-line FTP xfers
* fix fetch_ftp() so that hcode parsing is not done for file:// urls
(a } in the wrong place, and code at the wrong indent level...)
* change outfile to being a global (so it gets correctly reset)
* change parse_url to not remove leading '/' for non ftp urls.
whilst this is not totally rfc1738 compliant, other code kinda
assumes this is the case, and it doesn't hurt
 1.38 08-Mar-1999  lukem * add url_decode() - `in-place' decode %xx escapes in a given url component
* parse_url()
- only look for user[:pass] for an ftp url (per rfc1738)
- strip leading /'s in an ftp url. (almost per rfc 1738)
* fetch_url()
- decode a copy of the path and use that to build local filenames
- send port in http Host: header (suggested by cgd@netbsd.org)
* fetch_ftp()
- url_decode() the user, pass and path
- fix splitting of path into dir & file (partially from [bin/7073])
- don't bother caching the last host; it can cause problems when
using ftp:// transfers, or when the user changes between xfers
* improve documentation of auto-fetched url arguments (especially regarding
escape sequences in ftp:// urls)
* some whitespace & copyright updates

this should fix [bin/7073] William O Ferry <woferry@warp.wofme.com>,
as well as the metaissues raised in that PR.
 1.37 24-Jan-1999  lukem * -v enables verbose & progress, -V disables both
* set setvbuf(ttyout, NULL, _IOLBF, 0) and remove a bunch of fflush(ttyout).
* use fwrite() instead of write() for progressmeter (don't intermix stdio
with non stdio ops)
 1.36 23-Jan-1999  lukem support -f, which forces a cache flush for http xfers using either
`Pragma: no-cache' (http/1.0) or `Cache-Control: no-cache' (http/1.1)
 1.35 27-Dec-1998  lukem be saner about the method to determine which $proxy to use
 1.34 28-Sep-1998  lukem * add transfer rate to progress meter. (suggested by hubert feyrer)
* fix up progress meter display for files once > ~20.47MB has been
transferred. (noted by darren reed)
* use xsignal() for SIGWINCH, so window resizes don't abort xfers on
drainbamaged systems.
* don't assume that TIOCGWINSZ returns a non zero width.
 1.33 08-Aug-1998  lukem need <signal.h> for signal....
 1.32 03-Aug-1998  lukem features:
* support $no_proxy, which is a comma or space separated list of
host[:port] elements for which proxying is to be disabled.
(asked for by cgd in [bin/5027])
* if $FTPANONPASS is defined, use that as the anon ftp password
(instead of "`whoami`@")
* allow http URL's without a filename as long as an output file
is specified.

other stuff:
* implement parse_url(), which breaks up a URL into its bits, and use.
* simplify url_get() and auto_fetch() to use parse_url() and to not
modify the supplied URL or a copy of it.
* implement xmalloc() and xstrdup(); error-checked malloc()/strdup()
* add more consistency to messages, quoting strings in output as `%s'
 1.31 04-Jun-1998  lukem some fixes & enhancements from openbsd's ftp, with extra fixes by me:
* default to passive with active fallback. $FTPMODE modifies this behaviour.
-A forces active connection.
* support '-o outfile' for auto-fetched files. outfile can be a file,
`-' (for stdout), or '|command' (to output each file through command).
* support '-r waittime', which retries the connection after waittime seconds
if it fails.
* fix 'page file' when restart is non-zero.
* try all ip-addresses of a host in a http fetch (as the normal ftp fetch
does).

XXX: a ``broken pipe'' error sometimes occurs with -o '|command';
i haven't tracked this down yet.
 1.30 21-Jan-1998  lukem enable el_parse for unknown commands (so editrc(5) commands can be used directly
 1.29 18-Jan-1998  lukem define MAX_IN_PORT_T and use instead of USHRT_MAX. suggested by cgd@netbsd.org
 1.28 18-Jan-1998  lukem * ensure buffer for username is initialised, so ^D on username prompt
doesn't use garbage for the username. from "Soren S. Jorvang" <soren@t.dk>
in [bin/4559]
* use in_port_t for ports, and USHRT_MAX instead of 0xffff
(from millert@openbsd.org)
* use `NULL' instead of `(.... *)0' where appropriate.
 1.27 12-Dec-1997  gwr Kill the "progress bar" stuff when compiled with -DSMALL
because progressmeter() uses floats which are unwanted
when this is built under src/distrib/utils/x_ftp
(Maybe -DNO_FLOAT would be better?)
 1.26 14-Oct-1997  christos branches: 1.26.2;
PR/4257: Jaromir Dolecek: Compensate for the libedit changes.
 1.25 23-Aug-1997  lukem support $TMPDIR
 1.24 18-Aug-1997  lukem bugs fixed:
* don't interpret '-' or '|' when a local filename is determined from
the remote name (i.e, in mget, and in get with only one argument).
This is implemented using an extra argument to recvrequest().
Fixes a major security hole.
* clean up memory leak when using globulize()
* clean up a couple of comments
* fix wording in TNF copyright

features added:
* support for TIS fwtk gate-ftp servers:
* read defaults from $FTPSERVER && $FTPSERVERPORT
* start in gate-ftp mode if invoked as 'gate-ftp'
* toggle or set with 'gate [host [port]]'
 1.23 20-Jul-1997  lukem * use RCSID() && COPYRIGHT() macros
* cleanup for WARNS=1 (including some ugly '(void)&var' bits wrapped in
#ifdef __GNUC__ to shut up gcc warnings WRT setjmp/longjmp)
* use strtol() instead of atoi(), and more extensively check result of
conversion
* use u_int16_t instead of short or int for TCP port addresses
 1.22 10-Jun-1997  lukem Don't use editing or progress bar if $TERM isn't set or is "dumb". Fixes
emacs ange-ftp. Suggested by Todd Miller <Todd.Miller@courtesan.com>,
tested by matt green <mrg@eterna.com.au>
 1.21 05-Apr-1997  lukem * support $ftp_proxy for ftp:// transfers [bin/3245]
* add "more" & "less" as synonyms for "page"
* move editline setup code into controlediting(), and call appropriately.
only setup setup terminal if going into interactive mode. inspired
by Todd Miller <todd.miller@courtesan.com>
 1.20 16-Mar-1997  lukem Fixes from <Todd.Miller@cs.colorado.edu>, with some cleanup and reworks by me:
* only echo "Passive mode" in verbose mode; scripts that use ftp
may get unwanted output otherwise
* disable progress bar and modification time preservation when
retreiving to a non-regular files. fixes progress bar getting in
way of "get file /dev/tty"
* setup el_init() et al if editing is set, not if fromatty.
TODO: migrate this to a function, and call if editing is turned on later
in the session. also implement edit_cleanup if editing is turned off
* call el_set() after setting SIGWINCH handler. This fixes the problem
when suspending in a non-cbreak shell (e.g, csh) would trash your tty mode.
* reset interactive mode correctly in auto_fetch() mget mode
 1.19 14-Mar-1997  christos SMALLFTP->SMALL; remove SMALL from Makefile
 1.18 13-Mar-1997  lukem Features:
* support remglobbing of auto_fetch arguments
* new flag - '-e'; disable editing
* "page file" == "get file |${PAGER-less}"

Bugfixes/cleanup:
* consistently use a trailing '.' on messages
* code cleanup, including buffer overrun fixes, use puts
and putchar in places, etc (inspired by OpenBSD mods)
* disable progress bar when local-file is a pipe or '-'
* skip \r in http headers
* fix remote ftpd slash bug more elegantly (so it works with ////)
* abort_remote(): check if cout==NULL before using it. should fix [bin/3273]
* fixed up cosmetic problems when complete_remote() generated errors from the
remote server (such as "no files found", "login with user and pass", ...)
done by adding extra argument to remglob(), which is a pointer to an error
buffer to put messages in rather than printing to stdout.
 1.17 01-Feb-1997  lukem [Yet Another Huge Ftp Commit - hopefully the last for a while,
barring any more little things people want added ...]

New features:
* progressmeter is now asynchronous, so "stalled" transfers can be
detected. "- stalled -" is displayed instead of the ETA in this case.
When the xfer resumes, the time that the xfer was stalled for is
factored out of the ETA. It is debatable whether this is better than
not factoring it out, but I like it this way (I.e, if it stalls for 8
seconds and the ETA was 30 seconds, when it resumes the ETA will still
be 30 seconds).
* verbosity can be disabled on the command line (-V), so that in auto-fetch
mode the only lines displayed will be a description of the file, and
the progress bar (if possible)
* if the screen is resized (and detected via the SIGWINCH signal), the
progress bar will rescale automatically.

Bugs fixed:
* progress bar will not use the last character on the line, as this can
cause problems on some terminals
* screen dimensions (via ioctl(TIOCWINSZ)) should use stdout not stdin
* progressmeter() used some vars before initialising them
* ^D will quit now. [fixes bin/3162]
* use hstrerror() to generate error message for host name lookup failure.
* use getcwd instead of getwd (it should have been OK, but why tempt fate?)
* auto-fetch transfers will always return a positive exit value upon failure
or interruption, relative to the file's position in argv[].
* remote completion of / will work, without putting a leading "///".
This is actually a bug in ftpd(1), where "NLST /" prefixes all names
with "//", but fixing every ftpd(1) is not an option...
 1.16 19-Jan-1997  lukem New features:
* Command line editing via editline(3) library.
* Context sensitive command and file completion, including remote files.

Enhancements to auto-fetch feature:
* Support for http:// URLs using the http protocol, including proxy HTTP
support via $htty_proxy if it's defined.
* The connection is kept open between successive files on the same host.
(obviously, this does not count for http requests.)
* Return value of ftp is 0 on no error, or the offset in argv[] of the file
which failed (i.e., argv[x] failed, ftp returns x).
* If the path in an ftp URL or classic format line has a trailing '/',
cd to the path and enter interactive mode. Fixes [bin/3011], albiet
requiring the user to help ftp in determining the operation.

Other changes:
* '-P port' works for normal ftp, and is the default for all classic style
auto-fetch transfers and for ftp URLs that don't specify the port.
(previously it would just work for the first xfer.)
* Some code moved into separated files along logical divisions.
* Editing and completion can be compiled out with -DSMALLFTP.
 1.15 29-Dec-1996  lukem * preserve modtime if size is 0 [bin/3040, Enami Tsugutomo]
* in autofetch mode, don't retry a file if the connection failed - skip it
and move to the next file. [bin/3051, Matt Green]
* only print error messages from SIZE or MDTM if the user used 'size' or
'modt' (respectively). prevents extraneous warnings during normal transfers
when connected to a site which doesn't support these (behaviour prior to
last commit), but still allows error feedback to specific user requests
for this info (which the last commit broke).
* 'account': only accept one optional argument
* if invoked as pftp, default to passive mode on (from FreeBSD)
* remove leading '0 ' in progress bar - looked ugly
* use warn instead of perror
* use strncpy when src isn't known to have safe length
* remglob(): use mkstemp() to prevent symlink games, and don't override
_PATH_TMP, use it as the prefix to the temp file
 1.14 06-Dec-1996  lukem functionality mods:
* implement 'progress bar/meter' (inspired by ncftp). use 'progress' to
toggle on. it will display current file size to 5 digits, automatically
determining suffix (up to 16384 P (petabytes) == 2^64).
* 'ls' now uses NLST (unadorned listing), a la older ftp clients. 'dir'
still does LIST (long listing). idea from John Nemeth <jnemeth@cue.bc.ca>

bug fixes:
* return first line of reply in reply_string[] from getreply(), instead
of last line. This fixes [bin/741] (parsing of SYST), and also means
that SIZE and MDTM messages will be parsed correctly if they're longer
than 1 line.
* parse URL-style auto-ftps that have no filename correctly
(e.g, ftp://host, ftp://host/, ftp://host/dir/). pointed out by
Jaromir Dolecek <dolecek@saruman.ics.muni.cz>
* pass the correct size array in 2nd arg of utimes() when setting the
modification time
 1.13 28-Nov-1996  lukem More features, some of which were inspired by changes that
friedman@gnu.ai.mit.edu (Noah Friedman) made to his modified ftp:
- implement "lpwd" - local pwd
- implement "preserve" - toggle preserving of file modification
times on retrieved files
- allow for explicit "on" or "off" arg to toggle commands
- "exit" synonym for "quit", "msend" synonym for "mput"
- in confirmation mode, allow 'a' (yes to rest of current command),
and 'p' (turn off prompt mode, as if 'prompt off' was done,
effective immediately)
- "modtime" returns time formatted as localtime, not GMT

Bug fixes:
- check for extraneous args on commands
- cleanup const usage, line formatting
- create 0 length temporary file in remglob() to prevent symlink games
(from OpenBSD)
- check length of filename of ~/.netrc (from OpenBSD)
 1.12 25-Nov-1996  lukem cleanups/bugfixes:
- don't echo 'ACCT' parameter when debugging (a la 'PASS')
- Fix checking of directory access for "/foo", the parent
directory is "/", not "" (from FreeBSD)
- remove trailing whitespace on lines
- add any missing NetBSD tags
- cleanups to man page, includinging sorting options description

feature additions:
- variable sized hash marks (from [bin/683], but done in the hash command
as an optional arg)
- more user-friendly transfer time printing (from FreeBSD, with mods)
- '-p' command line option to jump into PASV mode (closes [bin/2857],
but with an option rather than checking argv[0])
- SIGINFO support for printing xfer stats when sending/receiving requests
- '-P port' for changing the port to connect to (from thorpej@netbsd.org)
- '-a': bypass normal login, and try anonymous login (from OpenBSD
via thorpej)
- autofetch files via url (ftp://...) or "classic" (host:/file)
(from OpenBSD via thorpej)
- 'ftp' synonymous with 'open' (from FreeBSD)
 1.11 07-May-1996  pk Fence off NULL args in getcmd() (fixes/works around PR#824).
 1.10 15-Sep-1995  pk Avoid falling off the end of a fixed-length array (from John F. Woods; PR#1431).
 1.9 14-Sep-1995  cgd needs string.h for protos.
 1.8 08-Sep-1995  tls Sync with 4.4lite2
 1.7 24-Dec-1994  cgd needs string.h
 1.6 31-Aug-1994  mycroft Fix option parsing bug reported by Uwe Arndt, and another related bogon.
 1.5 29-Aug-1994  mycroft branches: 1.5.2;
Add RCS ids.
 1.4 25-Aug-1994  cgd passive mode support, from David Carrel <carrel@cisco.com>, and cleaned
up for the new ftp sources by me.
 1.3 25-Aug-1994  cgd clean up import, no local changes.
 1.2 01-Aug-1993  mycroft Add RCS identifiers.
 1.1 21-Mar-1993  cgd branches: 1.1.1; 1.1.2;
Initial revision
 1.1.2.1 25-Aug-1994  cgd import from 4.4-Lite
 1.1.1.2 05-Sep-1995  tls imported from 44lite2
 1.1.1.1 21-Mar-1993  cgd initial import of 386bsd-0.1 sources
 1.5.2.2 31-Aug-1994  cgd From trunk.
 1.5.2.1 29-Aug-1994  cgd file main.c was added on branch netbsd-1-0 on 1994-08-31 22:07:08 +0000
 1.26.2.2 10-Nov-1998  cgd pull up rev(s) 1.28-1.34 from trunk. (feyrer)
 1.26.2.1 14-Dec-1997  mellon Pull rev 1.27 up from trunk (gwr)
 1.40.2.2 25-Jun-1999  cgd pull up rev 1.43 from trunk. (cgd)
 1.40.2.1 22-Jun-1999  perry pullup 1.40->1.42 (lukem)
 1.50.2.1 27-Dec-1999  wrstuden Pull up to last week's -current.
 1.70.2.1 23-Jun-2000  minoura Sync w/ netbsd-1-5-base.
 1.90.2.9 24-Jul-2005  tron Pull up revision 1.99 (requested by lukem in ticket #606):
Add NO_USAGE and NO_DEBUG so that we can fit in the floppies again.
 1.90.2.8 24-Jul-2005  tron Pull up revision 1.98 (requested by lukem in ticket #606):
Need to #ifdef around declaration of ch with NO_EDITCOMPLETE so this compiles
on rescue/miniroot's
 1.90.2.7 24-Jul-2005  tron Pull up revision 1.97 (requested by lukem in ticket #606):
Implement:
int getline(FILE *stream, char *buf, size_t buflen, const char **errormsg)
Read a line from the FILE stream into buf/buflen using fgets(), so up
to buflen-1 chars will be read and the result will be NUL terminated.
If the line has a trailing newline it will be removed.
If the line is too long, excess characters will be read until
newline/EOF/error.
Various -ve return values indicate different errors, and errormsg
will be changed to an error description if it's not NULL.
Convert to use getline() instead of fgets() whenever reading user input
to ensure that an overly long input line doesn't leave excess characters
for the next input operation to accidentally use as input.
Zero out the password & account after we've finished with it.
Consistently use getpass(3) (i.e, character echo suppressed) when
reading the account data. For some reason, historically the "login"
code suppressed echo for Account: yet the "user" command did not!
Display the hostname in the "getaddrinfo failed" warning.
Appease some -Wcast-qual warnings. Fixing all of these requires
significant code refactoring. (mmm, legacy code).
 1.90.2.6 24-Jul-2005  tron Pull up revision 1.96 (requested by lukem in ticket #606):
Make this compile on sparc64 (size_t != int).
 1.90.2.5 24-Jul-2005  tron Pull up revision 1.95 (requested by lukem in ticket #606):
Use size_t instead of int where appropriate.
 1.90.2.4 18-May-2005  snj Pull up revision 1.94 (requested by lukem in ticket #301):
* Correct the "optlen" argument passed to getsockopt(3) and setsockopt(3)
in various places. Fixes a problem noted by Allen Briggs.
* Improve warning printed when connect(2) for the data channel fails.
 1.90.2.3 18-May-2005  snj Pull up revision 1.93 (requested by lukem in ticket #318):
Use socklen_t instead of int as the 5th argument to getsockopt().
Improve invocation of setsockopt() and associated failure messages.
 1.90.2.2 11-May-2005  tron Pull up revision 1.92 (requested by lukem in ticket #289):
Prevent an overly-long input line causing a core dump when editing is enabled.
Issue noted by Ryoji Kanai in FreeBSD Problem Report # 77158.
 1.90.2.1 09-May-2005  tron Pull up revision 1.91 (requested by lukem in ticket #266):
gratuitous whitespace cleanup (before someone else jumps the gun...)
 1.105.4.1 09-Jan-2008  matt sync with HEAD
 1.106.6.1 18-May-2008  yamt sync with head.
 1.108.2.3 24-Sep-2008  wrstuden Merge in changes between wrstuden-revivesa-base-2 and
wrstuden-revivesa-base-3.
 1.108.2.2 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.108.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.113.6.1 13-May-2009  jym Sync with HEAD.

Third (and last) commit. See http://mail-index.netbsd.org/source-changes/2009/05/13/msg221222.html
 1.119.2.2 23-Jan-2013  yamt sync with head
 1.119.2.1 17-Apr-2012  yamt sync with head
 1.120.6.1 25-Feb-2013  tls resync with head
 1.120.2.2 27-Aug-2016  bouyer Apply patch, requested by nonaka in ticket #1375:
src/usr.bin/ftp/cmds.c: patch
src/usr.bin/ftp/fetch.c: patch
src/usr.bin/ftp/ftp.1: patch
src/usr.bin/ftp/ftp.c: patch
src/usr.bin/ftp/ftp_var.h: patch
src/usr.bin/ftp/main.c: patch
src/usr.bin/ftp/ssl.c: patch
src/usr.bin/ftp/ssl.h: patch
src/usr.bin/ftp/version.h: patch
Update ftp(1) to version 20150912, adding https via proxy support.
 1.120.2.1 17-Dec-2013  bouyer Apply patch, requested by tron in ticket #997:
usr.bin/ftp/Makefile patch
usr.bin/ftp/cmds.c patch
usr.bin/ftp/cmdtab.c patch
usr.bin/ftp/extern.h patch
usr.bin/ftp/fetch.c patch
usr.bin/ftp/ftp.1 patch
usr.bin/ftp/ftp.c patch
usr.bin/ftp/ftp_var.h patch
usr.bin/ftp/main.c patch
usr.bin/ftp/progressbar.c patch
usr.bin/ftp/ssl.c patch
usr.bin/ftp/ssl.h patch
usr.bin/ftp/util.c patch
usr.bin/ftp/version.h patch

Add HTTPS support to ftp(1).
 1.122.8.1 05-Nov-2015  riz Pull up following revision(s) (requested by wiz in ticket #981):
usr.bin/ftp/ftp.1: revision 1.135
usr.bin/ftp/ssl.c: revision 1.3
usr.bin/ftp/ssl.c: revision 1.4
usr.bin/ftp/ssl.h: revision 1.3
usr.bin/ftp/version.h: revision 1.86
usr.bin/ftp/version.h: revision 1.87
usr.bin/ftp/fetch.c: revision 1.207
usr.bin/ftp/main.c: revision 1.123
Add -x xferbufsize to set xferbuf size.
Implement -x xferbufsize set the socket send and receive buffer size,
as per 'xferbuf' in interactive mode.
Patch from Nicholas Mills (via private mail), with minor adjustment by me.
Add Server Name Indication (SNI) support for https.
Needed for e.g. some github URLs.
Bump version for SNI support.
servername cannot be NULL here.
Noted by joerg@.
 1.123.8.3 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1763:

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/cmds.c up to 1.141
usr.bin/ftp/complete.c up to 1.47
usr.bin/ftp/domacro.c up to 1.23
usr.bin/ftp/extern.h up to 1.82
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp.c up to 1.174
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/progressbar.c up to 1.24
usr.bin/ftp/progressbar.h up to 1.9
usr.bin/ftp/ssl.c up to 1.12
usr.bin/ftp/ssl.h up to 1.5
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

ftp(1): validate address from PASV and LPSV response.
ftp(1): use raw write(2) instead of fwrite(3) to avoid stream
corruption because of the progress bar interrupts.
Fixes for PR 56219 and PR 55857.
PR 57003: Support relative redirects.
 1.123.8.2 12-Sep-2022  martin Backout ticket #1763 for now - trust anchors are not solved.
 1.123.8.1 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1763:

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/cmds.c up to 1.141
usr.bin/ftp/complete.c up to 1.47
usr.bin/ftp/domacro.c up to 1.23
usr.bin/ftp/extern.h up to 1.82
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp.c up to 1.174
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/progressbar.c up to 1.24
usr.bin/ftp/progressbar.h up to 1.9
usr.bin/ftp/ssl.c up to 1.11
usr.bin/ftp/ssl.h up to 1.5
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

ftp(1): validate address from PASV and LPSV response.
ftp(1): use raw write(2) instead of fwrite(3) to avoid stream
corruption because of the progress bar interrupts.
Fixes for PR 56219 and PR 55857.
PR 57003: Support relative redirects.
 1.125.4.1 10-Jun-2019  christos Sync with HEAD
 1.126.2.3 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1523

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/ssl.c up to 1.12
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

PR 57003: Support relative redirects.
 1.126.2.2 12-Sep-2022  martin Back out ticket #1523 for now - trust anchor validation is not solved.
 1.126.2.1 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1523

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/ssl.c up to 1.11
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

PR 57003: Support relative redirects.
 1.128.2.3 02-Dec-2024  martin Pull up following revision(s) (requested by lukem in ticket #1021):

usr.bin/ftp/ftp.c: revision 1.178
usr.bin/ftp/version.h: revision 1.99
usr.bin/ftp/ruserpass.c: revision 1.35
usr.bin/ftp/main.c: revision 1.134
usr.bin/ftp/main.c: revision 1.135
usr.bin/ftp/progressbar.c: revision 1.27
usr.bin/ftp/util.c: revision 1.165
usr.bin/ftp/cmds.c: revision 1.144
usr.bin/ftp/extern.h: revision 1.84
usr.bin/ftp/fetch.c: revision 1.242
usr.bin/ftp/ftp.1: revision 1.160

s/bninary/binary/ in comment.

extract duplicate code into a function.

Check bounds when copying to destination.

Remove const where the const string ended up being overwritten.

use unsigned when doing shifts.

remove const when string gets overwritten.

ftp: exit non-zero if short http transfer when filesize is known
If a http file size is known and the fetch finishes with less bytes
transferred, exit non-zero.
Bump version to 20241129.
PR bin/54713
PR bin/58281

ftp: help improvements
Document -? as a separate mode.
Document -H HEADER in the usage.
Clarify units for -b and -x.
Consistent argument names between ftp -? and ftp(1).

ftp: order getopt Upper before lower
Consistently order options in getopt and the switch
with the upper case option before the lower case option.
This makes it easier to cross-reference with -? and ftp(1).
No functional change.
 1.128.2.2 13-Oct-2024  martin Pull up following revision(s) (requested by riastradh in ticket #970):

tests/usr.bin/Makefile: revision 1.42
usr.bin/ftp/ruserpass.c: revision 1.34
usr.bin/ftp/main.c: revision 1.130
usr.bin/ftp/ssl.c: revision 1.17
usr.bin/ftp/main.c: revision 1.131
usr.bin/ftp/ssl.c: revision 1.18
usr.bin/ftp/main.c: revision 1.132
usr.bin/ftp/ssl.c: revision 1.19
usr.bin/ftp/main.c: revision 1.133
distrib/sets/lists/tests/mi: revision 1.1342
usr.bin/ftp/ftp.1: revision 1.151
usr.bin/ftp/ftp.1: revision 1.152
usr.bin/ftp/progressbar.c: revision 1.25
usr.bin/ftp/ftp.1: revision 1.153
usr.bin/ftp/progressbar.c: revision 1.26
usr.bin/ftp/ftp.1: revision 1.155
usr.bin/ftp/ftp.1: revision 1.156
usr.bin/ftp/fetch.c: revision 1.239
usr.bin/ftp/ftp.1: revision 1.157
usr.bin/ftp/ftp.1: revision 1.158
usr.bin/ftp/ftp.1: revision 1.159
usr.bin/ftp/ftp_var.h: revision 1.87
etc/mtree/NetBSD.dist.tests: revision 1.208
usr.bin/ftp/ftp_var.h: revision 1.88
usr.bin/ftp/ftp_var.h: revision 1.89
usr.bin/ftp/cmds.c: revision 1.142
usr.bin/ftp/util.c: revision 1.168
usr.bin/ftp/cmds.c: revision 1.143
tests/usr.bin/ftp/custom_headers.sh: revision 1.1
usr.bin/ftp/ssl.c: revision 1.20
usr.bin/ftp/complete.c: revision 1.48
tests/usr.bin/ftp/Makefile: revision 1.1
tests/usr.bin/ftp/t_custom_headers.sh: revision 1.1
usr.bin/ftp/fetch.c: revision 1.240
usr.bin/ftp/fetch.c: revision 1.241
usr.bin/ftp/ftp.c: revision 1.176
usr.bin/ftp/ftp.c: revision 1.177
(all via patch)

ftp(1): wording and formatting improvements

Fix grammar issue with "Support values" reported in private mail.
Document all file transfer types in "type" and cross-reference that.
Consistency fixes in describing file transfer parameters and types.

Fix some mandoc -Tlint issues (except "useless macro: Tn").

Add -b <buflen> to specify the buffer size.

ftp: bump FTPBUFLEN from 4kB to 16kB
sourceforge.net returns a 5kB content-security-policy.
Analyzed by mlelstv@ who reports usual limits are between 4kB and 48kB.
default is now 16K

ftp: improve -b documentation

Order -b bufsize in the synopsis.

Document the actual default value.

ftp: improve units used in comments and errors
Use "KiB" instead of "K" in errors.
Clarify related comments.

pass some lint.

PR/58581: Sunil Nimmagadda: Add flag to allow specifying extra http header
fields.

ftp(1): Nix trailing whitespace in man page.
No functional change intended.

PR bin/58581: ftp(1) should allow specifying header fields in http requests
fix markup (h -> H), explain about multiple headers, fix usage (from RVP)

Don't forget the dot, use the intended macro name (I think),
and improve the wording a little. (All related to the -H option.)
ftp(1): Add test for custom HTTP header fields.

Based on a patch from Sunil Nimmagadda.

PR bin/58581: ftp(1) should allow specifying header fields in http
requests
 1.128.2.1 16-May-2023  martin Pull up following revision(s) (requested by lukem in ticket #170):

usr.bin/ftp/ssl.c: revision 1.13
usr.bin/ftp/fetch.c: revision 1.236
usr.bin/ftp/util.c: revision 1.166
usr.bin/ftp/main.c: revision 1.129
usr.bin/ftp/extern.h: revision 1.83
usr.bin/ftp/ftp.1: revision 1.148
usr.bin/ftp/cmdtab.c: revision 1.53
usr.bin/ftp/version.h: revision 1.96

Add option sslnoverify to control validation of SSL certificates.

Add netrc processing to fetch-mode (URL on command line) to enable options and autologin
via netrc.

Fix SSL cleanup in some error paths.

Certificate validation is now enabled by default. Set FTPSSLNOVERIFY=1 in environment
or configure a corresponding init macro via netrc to not validate certs (required if
you haven't installed a required CA certificate for OpenSSL).

Discussed with lukem@ on icb.
 1.130.2.1 02-Aug-2025  perseant Sync with HEAD
 1.8 01-Oct-1999  lukem * don't bother with "pathnames.h"; it contained just a single #define.
instead, include <paths.h> in the files that need it
* slightly reorg ftp_var.h
 1.7 09-Jan-1997  tls branches: 1.7.6;
RCS ID police
 1.6 29-Dec-1996  lukem * preserve modtime if size is 0 [bin/3040, Enami Tsugutomo]
* in autofetch mode, don't retry a file if the connection failed - skip it
and move to the next file. [bin/3051, Matt Green]
* only print error messages from SIZE or MDTM if the user used 'size' or
'modt' (respectively). prevents extraneous warnings during normal transfers
when connected to a site which doesn't support these (behaviour prior to
last commit), but still allows error feedback to specific user requests
for this info (which the last commit broke).
* 'account': only accept one optional argument
* if invoked as pftp, default to passive mode on (from FreeBSD)
* remove leading '0 ' in progress bar - looked ugly
* use warn instead of perror
* use strncpy when src isn't known to have safe length
* remglob(): use mkstemp() to prevent symlink games, and don't override
_PATH_TMP, use it as the prefix to the temp file
 1.5 08-Sep-1995  tls Sync with 4.4lite2
 1.4 29-Aug-1994  mycroft branches: 1.4.2;
Add RCS ids.
 1.3 25-Aug-1994  cgd clean up import, no local changes.
 1.2 01-Aug-1993  mycroft Add RCS identifiers.
 1.1 21-Mar-1993  cgd branches: 1.1.1; 1.1.2;
Initial revision
 1.1.2.1 25-Aug-1994  cgd import from 4.4-Lite
 1.1.1.2 05-Sep-1995  tls imported from 44lite2
 1.1.1.1 21-Mar-1993  cgd initial import of 386bsd-0.1 sources
 1.4.2.2 29-Aug-1994  mycroft Add RCS ids.
 1.4.2.1 29-Aug-1994  mycroft file pathnames.h was added on branch netbsd-1-0 on 1994-08-29 03:09:20 +0000
 1.7.6.1 27-Dec-1999  wrstuden Pull up to last week's -current.
 1.27 04-Oct-2024  christos use unsigned when doing shifts.
 1.26 25-Sep-2024  christos pass some lint.
 1.25 19-Jul-2024  lukem ftp: improve units used in comments and errors

Use "KiB" instead of "K" in errors.
Clarify related comments.
 1.24 06-Jan-2021  lukem branches: 1.24.6; 1.24.8;
ftp: don't use restartable signals

Refactor to not rely upon restartable signals (SA_RESTART),
possibly fixing intermittent failures with -q QUITTIME.

ftp transfers: handle EINTR/EAGAIN in copy_bytes(),
instead of relying upon restartable signals.

http/https transfers: Explicitly print an error similar to
progressmeter() when timing-out for -Q QUITTIME in fetch_wait(),
and set errno to ETIMEDOUT so that the warn() in fetch_url()
prints a more accurate error message.

PR/55857
 1.23 22-Jun-2019  christos branches: 1.23.2;
make this compile again with -DNO_PROGRESS
 1.22 27-Jun-2012  riastradh branches: 1.22.24; 1.22.32;
Clean up #includes in progress(1).

Sort them, omit needless ones, and add omitted needful ones.

Omissions revealed by pilfering the code and trying to build it on
Mac OS X. We ought to have a better way to reveal these omissions...
 1.21 12-Apr-2009  lukem branches: 1.21.6; 1.21.8;
Fix numerous WARNS=4 issues (-Wcast-qual -Wsign-compare).
 1.20 30-Sep-2008  lukem branches: 1.20.6;
update copyrights
 1.19 28-Apr-2008  martin Remove clause 3 and 4 from TNF licenses
 1.18 21-Jan-2008  lukem branches: 1.18.4;
Turn off the alarmtimer before resetting the SIGALRM handler back to SIG_DFL.
Fix from dieter roelants <dieter.NetBSD@pandora.be>
Should fix PR 35630.
 1.17 05-May-2007  martin branches: 1.17.4;
const-as-const-can: make the "suffixes" pointers const, as probably was
intended, but done wrong originally (redundant const was removed
per PR bin/36280)
 1.16 05-May-2007  christos PR/36280: Christer Folkesson: Remove extra const
 1.15 17-Apr-2007  lukem * Implement -s srcaddr; uses srcaddr as the local IP address for all
connections.
Based on code in the version of ftp that FreeBSD had before they
replaced it with lukemftp.
* Move error message handling into ftp_connect() rather than in the
caller, so that more specific error reporting can occur.
* Improve consistency of various warning and error messages.
 1.14 12-Apr-2007  lukem Use the IEC 60027-2 2^n based "KiB", "MiB", "GiB", (etc)
suffixes for byte displays since we're using 2^n scaling,
instead of the SI 10^n based "KB", "MB", "GB" (etc).
 1.13 01-May-2006  christos Coverity CID 1447: Avoid buffer overflow.
 1.12 01-May-2006  christos Coverity CID 1448: Avoid static array overflow.
 1.11 19-Jul-2005  lukem Revert back to using an int (instead of size_t) for holding a value that may
be negative.
Fixes progressbar display on terminals <43 columns wide.
Bug noted and solution suggested by Gavan Fantom.
 1.10 09-Jun-2005  lukem copyright maintenance
 1.9 20-May-2005  he Printf field widths and size_t don't always mix well, so cast to int.
Fixes build problem for alpha.
 1.8 19-May-2005  lukem Use size_t instead of int where appropriate.
 1.7 11-Apr-2005  lukem gratuitous whitespace cleanup (before someone else jumps the gun...)
 1.6 10-Feb-2005  jmc branches: 1.6.2;
Fix compilation issues for NO_PROGRESS
 1.5 09-Mar-2004  hubertf Allow custom text printed before (left of) the progress bar from progress(1):

miyu# cat openoffice-linux-1.1.0.tgz | progress -z -p 'Bytes written: ' dd of=/dev/null bs=1m
Bytes written: 193 MB 13.83 MB/s 0+195211 records in
and:
miyu# progress -f openoffice-linux-1.1.0.tgz -z -p 'Bytes written: ' dd of=/dev/null bs=1m
Bytes written: 28% |****** | 57919 KB 14.12 MB/s 00:09 ETA

OK'd by lukem.
 1.4 17-Jul-2003  lukem tweak copyright; the stuff jason did in util.c wasn't migrated to this file.
 1.3 28-Feb-2003  lukem Crank dates on TNF copyright.
Remove UCB & WIDE copyrights; the progressbar functionality was
written entirely by me under the TNF copyright.
 1.2 12-Feb-2003  grant change a couple #ifdef's so /usr/bin/progress doesn't stomp all over
the tty if it's not in the foreground.

ok'd by jhawk.
 1.1 21-Jan-2003  jhawk branches: 1.1.2;
Abstract out progress bar support into progressbar.[ch], for inclusion
in external programs (conditionalized on -DSTANDALONE_PROGRESS).

The following moved from util.c to progressbar.c:
alarmtimer(), progressmeter(), psummary(), ptransfer(),
xsignal(), xsignal_restart()

The following moved from extern.h and ftp_var.h to progressbar.h:
STALLTIME, verbose, fromatty, progress, quit_time, ttywidth
 1.1.2.2 21-Jan-2003  jhawk Abstract out progress bar support into progressbar.[ch], for inclusion
in external programs (conditionalized on -DSTANDALONE_PROGRESS).

The following moved from util.c to progressbar.c:
alarmtimer(), progressmeter(), psummary(), ptransfer(),
xsignal(), xsignal_restart()

The following moved from extern.h and ftp_var.h to progressbar.h:
STALLTIME, verbose, fromatty, progress, quit_time, ttywidth
 1.1.2.1 21-Jan-2003  jhawk file progressbar.c was added on branch netbsd-1-6 on 2003-01-21 16:08:09 +0000
 1.6.2.5 24-Jul-2005  tron Pull up revision 1.11 (requested by lukem in ticket #606):
Revert back to using an int (instead of size_t) for holding a value that may
be negative.
Fixes progressbar display on terminals <43 columns wide.
Bug noted and solution suggested by Gavan Fantom.
 1.6.2.4 24-Jul-2005  tron Pull up revision 1.10 (requested by lukem in ticket #606):
copyright maintenance
 1.6.2.3 24-Jul-2005  tron Pull up revision 1.9 (requested by lukem in ticket #606):
Printf field widths and size_t don't always mix well, so cast to int.
Fixes build problem for alpha.
 1.6.2.2 24-Jul-2005  tron Pull up revision 1.8 (requested by lukem in ticket #606):
Use size_t instead of int where appropriate.
 1.6.2.1 09-May-2005  tron Pull up revision 1.7 (requested by lukem in ticket #266):
gratuitous whitespace cleanup (before someone else jumps the gun...)
 1.17.4.1 23-Mar-2008  matt sync with HEAD
 1.18.4.1 18-May-2008  yamt sync with head.
 1.20.6.1 13-May-2009  jym Sync with HEAD.

Third (and last) commit. See http://mail-index.netbsd.org/source-changes/2009/05/13/msg221222.html
 1.21.8.1 17-Dec-2013  bouyer Apply patch, requested by tron in ticket #997:
usr.bin/ftp/Makefile patch
usr.bin/ftp/cmds.c patch
usr.bin/ftp/cmdtab.c patch
usr.bin/ftp/extern.h patch
usr.bin/ftp/fetch.c patch
usr.bin/ftp/ftp.1 patch
usr.bin/ftp/ftp.c patch
usr.bin/ftp/ftp_var.h patch
usr.bin/ftp/main.c patch
usr.bin/ftp/progressbar.c patch
usr.bin/ftp/ssl.c patch
usr.bin/ftp/ssl.h patch
usr.bin/ftp/util.c patch
usr.bin/ftp/version.h patch

Add HTTPS support to ftp(1).
 1.21.6.1 30-Oct-2012  yamt sync with head
 1.22.32.1 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.22.24.3 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1763:

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/cmds.c up to 1.141
usr.bin/ftp/complete.c up to 1.47
usr.bin/ftp/domacro.c up to 1.23
usr.bin/ftp/extern.h up to 1.82
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp.c up to 1.174
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/progressbar.c up to 1.24
usr.bin/ftp/progressbar.h up to 1.9
usr.bin/ftp/ssl.c up to 1.12
usr.bin/ftp/ssl.h up to 1.5
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

ftp(1): validate address from PASV and LPSV response.
ftp(1): use raw write(2) instead of fwrite(3) to avoid stream
corruption because of the progress bar interrupts.
Fixes for PR 56219 and PR 55857.
PR 57003: Support relative redirects.
 1.22.24.2 12-Sep-2022  martin Backout ticket #1763 for now - trust anchors are not solved.
 1.22.24.1 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1763:

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/cmds.c up to 1.141
usr.bin/ftp/complete.c up to 1.47
usr.bin/ftp/domacro.c up to 1.23
usr.bin/ftp/extern.h up to 1.82
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp.c up to 1.174
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/progressbar.c up to 1.24
usr.bin/ftp/progressbar.h up to 1.9
usr.bin/ftp/ssl.c up to 1.11
usr.bin/ftp/ssl.h up to 1.5
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

ftp(1): validate address from PASV and LPSV response.
ftp(1): use raw write(2) instead of fwrite(3) to avoid stream
corruption because of the progress bar interrupts.
Fixes for PR 56219 and PR 55857.
PR 57003: Support relative redirects.
 1.23.2.2 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1523

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/ssl.c up to 1.12
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

PR 57003: Support relative redirects.
 1.23.2.1 29-Jan-2021  martin Pull up following revision(s) (requested by lukem in ticket #1190):

usr.bin/ftp/progressbar.c: revision 1.24
usr.bin/ftp/ssl.c: revision 1.9
usr.bin/ftp/progressbar.h: revision 1.9
usr.bin/ftp/ftp.c: revision 1.171
usr.bin/ftp/version.h: revision 1.92

ftp: don't use restartable signals

Refactor to not rely upon restartable signals (SA_RESTART),
possibly fixing intermittent failures with -q QUITTIME.
ftp transfers: handle EINTR/EAGAIN in copy_bytes(),
instead of relying upon restartable signals.

http/https transfers: Explicitly print an error similar to
progressmeter() when timing-out for -Q QUITTIME in fetch_wait(),
and set errno to ETIMEDOUT so that the warn() in fetch_url()
prints a more accurate error message.

PR/55857
 1.24.8.1 02-Aug-2025  perseant Sync with HEAD
 1.24.6.2 02-Dec-2024  martin Pull up following revision(s) (requested by lukem in ticket #1021):

usr.bin/ftp/ftp.c: revision 1.178
usr.bin/ftp/version.h: revision 1.99
usr.bin/ftp/ruserpass.c: revision 1.35
usr.bin/ftp/main.c: revision 1.134
usr.bin/ftp/main.c: revision 1.135
usr.bin/ftp/progressbar.c: revision 1.27
usr.bin/ftp/util.c: revision 1.165
usr.bin/ftp/cmds.c: revision 1.144
usr.bin/ftp/extern.h: revision 1.84
usr.bin/ftp/fetch.c: revision 1.242
usr.bin/ftp/ftp.1: revision 1.160

s/bninary/binary/ in comment.

extract duplicate code into a function.

Check bounds when copying to destination.

Remove const where the const string ended up being overwritten.

use unsigned when doing shifts.

remove const when string gets overwritten.

ftp: exit non-zero if short http transfer when filesize is known
If a http file size is known and the fetch finishes with less bytes
transferred, exit non-zero.
Bump version to 20241129.
PR bin/54713
PR bin/58281

ftp: help improvements
Document -? as a separate mode.
Document -H HEADER in the usage.
Clarify units for -b and -x.
Consistent argument names between ftp -? and ftp(1).

ftp: order getopt Upper before lower
Consistently order options in getopt and the switch
with the upper case option before the lower case option.
This makes it easier to cross-reference with -? and ftp(1).
No functional change.
 1.24.6.1 13-Oct-2024  martin Pull up following revision(s) (requested by riastradh in ticket #970):

tests/usr.bin/Makefile: revision 1.42
usr.bin/ftp/ruserpass.c: revision 1.34
usr.bin/ftp/main.c: revision 1.130
usr.bin/ftp/ssl.c: revision 1.17
usr.bin/ftp/main.c: revision 1.131
usr.bin/ftp/ssl.c: revision 1.18
usr.bin/ftp/main.c: revision 1.132
usr.bin/ftp/ssl.c: revision 1.19
usr.bin/ftp/main.c: revision 1.133
distrib/sets/lists/tests/mi: revision 1.1342
usr.bin/ftp/ftp.1: revision 1.151
usr.bin/ftp/ftp.1: revision 1.152
usr.bin/ftp/progressbar.c: revision 1.25
usr.bin/ftp/ftp.1: revision 1.153
usr.bin/ftp/progressbar.c: revision 1.26
usr.bin/ftp/ftp.1: revision 1.155
usr.bin/ftp/ftp.1: revision 1.156
usr.bin/ftp/fetch.c: revision 1.239
usr.bin/ftp/ftp.1: revision 1.157
usr.bin/ftp/ftp.1: revision 1.158
usr.bin/ftp/ftp.1: revision 1.159
usr.bin/ftp/ftp_var.h: revision 1.87
etc/mtree/NetBSD.dist.tests: revision 1.208
usr.bin/ftp/ftp_var.h: revision 1.88
usr.bin/ftp/ftp_var.h: revision 1.89
usr.bin/ftp/cmds.c: revision 1.142
usr.bin/ftp/util.c: revision 1.168
usr.bin/ftp/cmds.c: revision 1.143
tests/usr.bin/ftp/custom_headers.sh: revision 1.1
usr.bin/ftp/ssl.c: revision 1.20
usr.bin/ftp/complete.c: revision 1.48
tests/usr.bin/ftp/Makefile: revision 1.1
tests/usr.bin/ftp/t_custom_headers.sh: revision 1.1
usr.bin/ftp/fetch.c: revision 1.240
usr.bin/ftp/fetch.c: revision 1.241
usr.bin/ftp/ftp.c: revision 1.176
usr.bin/ftp/ftp.c: revision 1.177
(all via patch)

ftp(1): wording and formatting improvements

Fix grammar issue with "Support values" reported in private mail.
Document all file transfer types in "type" and cross-reference that.
Consistency fixes in describing file transfer parameters and types.

Fix some mandoc -Tlint issues (except "useless macro: Tn").

Add -b <buflen> to specify the buffer size.

ftp: bump FTPBUFLEN from 4kB to 16kB
sourceforge.net returns a 5kB content-security-policy.
Analyzed by mlelstv@ who reports usual limits are between 4kB and 48kB.
default is now 16K

ftp: improve -b documentation

Order -b bufsize in the synopsis.

Document the actual default value.

ftp: improve units used in comments and errors
Use "KiB" instead of "K" in errors.
Clarify related comments.

pass some lint.

PR/58581: Sunil Nimmagadda: Add flag to allow specifying extra http header
fields.

ftp(1): Nix trailing whitespace in man page.
No functional change intended.

PR bin/58581: ftp(1) should allow specifying header fields in http requests
fix markup (h -> H), explain about multiple headers, fix usage (from RVP)

Don't forget the dot, use the intended macro name (I think),
and improve the wording a little. (All related to the -H option.)
ftp(1): Add test for custom HTTP header fields.

Based on a patch from Sunil Nimmagadda.

PR bin/58581: ftp(1) should allow specifying header fields in http
requests
 1.9 06-Jan-2021  lukem ftp: don't use restartable signals

Refactor to not rely upon restartable signals (SA_RESTART),
possibly fixing intermittent failures with -q QUITTIME.

ftp transfers: handle EINTR/EAGAIN in copy_bytes(),
instead of relying upon restartable signals.

http/https transfers: Explicitly print an error similar to
progressmeter() when timing-out for -Q QUITTIME in fetch_wait(),
and set errno to ETIMEDOUT so that the warn() in fetch_url()
prints a more accurate error message.

PR/55857
 1.8 12-Apr-2009  lukem branches: 1.8.38; 1.8.48;
Fix numerous WARNS=4 issues (-Wcast-qual -Wsign-compare).
 1.7 30-Sep-2008  lukem branches: 1.7.6;
update copyrights
 1.6 28-Apr-2008  martin Remove clause 3 and 4 from TNF licenses
 1.5 10-Feb-2005  jmc branches: 1.5.26;
White space nit
 1.4 09-Mar-2004  hubertf Allow custom text printed before (left of) the progress bar from progress(1):

miyu# cat openoffice-linux-1.1.0.tgz | progress -z -p 'Bytes written: ' dd of=/dev/null bs=1m
Bytes written: 193 MB 13.83 MB/s 0+195211 records in
and:
miyu# progress -f openoffice-linux-1.1.0.tgz -z -p 'Bytes written: ' dd of=/dev/null bs=1m
Bytes written: 28% |****** | 57919 KB 14.12 MB/s 00:09 ETA

OK'd by lukem.
 1.3 28-Feb-2003  lukem Crank dates on TNF copyright.
Remove UCB & WIDE copyrights; the progressbar functionality was
written entirely by me under the TNF copyright.
 1.2 12-Feb-2003  grant change a couple #ifdef's so /usr/bin/progress doesn't stomp all over
the tty if it's not in the foreground.

ok'd by jhawk.
 1.1 21-Jan-2003  jhawk branches: 1.1.2;
Abstract out progress bar support into progressbar.[ch], for inclusion
in external programs (conditionalized on -DSTANDALONE_PROGRESS).

The following moved from util.c to progressbar.c:
alarmtimer(), progressmeter(), psummary(), ptransfer(),
xsignal(), xsignal_restart()

The following moved from extern.h and ftp_var.h to progressbar.h:
STALLTIME, verbose, fromatty, progress, quit_time, ttywidth
 1.1.2.2 21-Jan-2003  jhawk Abstract out progress bar support into progressbar.[ch], for inclusion
in external programs (conditionalized on -DSTANDALONE_PROGRESS).

The following moved from util.c to progressbar.c:
alarmtimer(), progressmeter(), psummary(), ptransfer(),
xsignal(), xsignal_restart()

The following moved from extern.h and ftp_var.h to progressbar.h:
STALLTIME, verbose, fromatty, progress, quit_time, ttywidth
 1.1.2.1 21-Jan-2003  jhawk file progressbar.h was added on branch netbsd-1-6 on 2003-01-21 16:08:09 +0000
 1.5.26.1 18-May-2008  yamt sync with head.
 1.7.6.1 13-May-2009  jym Sync with HEAD.

Third (and last) commit. See http://mail-index.netbsd.org/source-changes/2009/05/13/msg221222.html
 1.8.48.2 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1523

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/ssl.c up to 1.12
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

PR 57003: Support relative redirects.
 1.8.48.1 29-Jan-2021  martin Pull up following revision(s) (requested by lukem in ticket #1190):

usr.bin/ftp/progressbar.c: revision 1.24
usr.bin/ftp/ssl.c: revision 1.9
usr.bin/ftp/progressbar.h: revision 1.9
usr.bin/ftp/ftp.c: revision 1.171
usr.bin/ftp/version.h: revision 1.92

ftp: don't use restartable signals

Refactor to not rely upon restartable signals (SA_RESTART),
possibly fixing intermittent failures with -q QUITTIME.
ftp transfers: handle EINTR/EAGAIN in copy_bytes(),
instead of relying upon restartable signals.

http/https transfers: Explicitly print an error similar to
progressmeter() when timing-out for -Q QUITTIME in fetch_wait(),
and set errno to ETIMEDOUT so that the warn() in fetch_url()
prints a more accurate error message.

PR/55857
 1.8.38.3 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1763:

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/cmds.c up to 1.141
usr.bin/ftp/complete.c up to 1.47
usr.bin/ftp/domacro.c up to 1.23
usr.bin/ftp/extern.h up to 1.82
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp.c up to 1.174
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/progressbar.c up to 1.24
usr.bin/ftp/progressbar.h up to 1.9
usr.bin/ftp/ssl.c up to 1.12
usr.bin/ftp/ssl.h up to 1.5
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

ftp(1): validate address from PASV and LPSV response.
ftp(1): use raw write(2) instead of fwrite(3) to avoid stream
corruption because of the progress bar interrupts.
Fixes for PR 56219 and PR 55857.
PR 57003: Support relative redirects.
 1.8.38.2 12-Sep-2022  martin Backout ticket #1763 for now - trust anchors are not solved.
 1.8.38.1 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1763:

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/cmds.c up to 1.141
usr.bin/ftp/complete.c up to 1.47
usr.bin/ftp/domacro.c up to 1.23
usr.bin/ftp/extern.h up to 1.82
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp.c up to 1.174
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/progressbar.c up to 1.24
usr.bin/ftp/progressbar.h up to 1.9
usr.bin/ftp/ssl.c up to 1.11
usr.bin/ftp/ssl.h up to 1.5
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

ftp(1): validate address from PASV and LPSV response.
ftp(1): use raw write(2) instead of fwrite(3) to avoid stream
corruption because of the progress bar interrupts.
Fixes for PR 56219 and PR 55857.
PR 57003: Support relative redirects.
 1.35 04-Oct-2024  christos extract duplicate code into a function.
 1.34 19-Jul-2024  lukem ftp: improve units used in comments and errors

Use "KiB" instead of "K" in errors.
Clarify related comments.
 1.33 17-Apr-2007  lukem branches: 1.33.86; 1.33.88;
* Implement -s srcaddr; uses srcaddr as the local IP address for all
connections.
Based on code in the version of ftp that FreeBSD had before they
replaced it with lukemftp.
* Move error message handling into ftp_connect() rather than in the
caller, so that more specific error reporting can occur.
* Improve consistency of various warning and error messages.
 1.32 12-Apr-2007  lukem Various fixes from Nicholas Marriott sent to openbsd-tech and in private email:
* Prevent segfault if a .netrc token is too long.
* Prevent segfault when using a macro from an empty macdef.
* Check more return values.
 1.31 31-Jan-2006  christos Rename xfoo() to ftp_foo() to avoid collisions with libssh. Don't ask.
 1.30 10-Jun-2005  lukem Implement:
int getline(FILE *stream, char *buf, size_t buflen, const char **errormsg)
Read a line from the FILE stream into buf/buflen using fgets(), so up
to buflen-1 chars will be read and the result will be NUL terminated.
If the line has a trailing newline it will be removed.
If the line is too long, excess characters will be read until
newline/EOF/error.
Various -ve return values indicate different errors, and errormsg
will be changed to an error description if it's not NULL.

Convert to use getline() instead of fgets() whenever reading user input
to ensure that an overly long input line doesn't leave excess characters
for the next input operation to accidentally use as input.

Zero out the password & account after we've finished with it.

Consistently use getpass(3) (i.e, character echo suppressed) when
reading the account data. For some reason, historically the "login"
code suppressed echo for Account: yet the "user" command did not!

Display the hostname in the "getaddrinfo failed" warning.

Appease some -Wcast-qual warnings. Fixing all of these requires
significant code refactoring. (mmm, legacy code).
 1.29 07-Aug-2003  agc branches: 1.29.6;
Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22365, verified by myself.
 1.28 15-Nov-2000  lukem - implement "mreget"; as per "mget" but uses "reget" instead of "get"
- add -N netrc and $NETRC, as methods to select an alternative .netrc file
- cache local user name and home directory for further use
- in mget(), use docase() instead of a local version to do the case
conversion.
 1.27 18-Jul-2000  lukem convert to new knf
 1.26 01-May-2000  lukem convert to ANSI KNF
 1.25 24-Oct-1999  lukem new features:
- add `usage'; displays the usage of a command.
implemented by calling the c_handler() with argc = 0, argv = "funcname".
- add `passive auto'; does the same as $FTPMODE=auto.
- add `set [option value]'; display all options, or set an option to a value.
- add `unset option'; unset an option.
- add getoptionvalue() to retrieve an option's value, and replace a few
global variables with calls to this.
- implement cleanuppeer(), which resets various bits of state back to
`disconnected'. call in disconnect() and lostpeer().
- support completing on `options'.
- improve recovery after a SIGINT may have closed the connection.
XXX: there's still a couple to fix

other stuff:
- various consistency fixes in the man page.
- ensure that the command usage strings in the code and man page match reality.
- mput/mget: check that the connection still exists before each xfer.
- minor cosmetic changes in confirm().
- set code correctly in sizecmd() and modtime()
- don't need \n in err() strings.
- change lostpeer to take an argument (rather than casting (sig_t)lostpeer
in signal handlers)
- knf and whitespace police.
 1.24 27-Sep-1999  lukem * idle(): rename to idlecmd(). certain linux distributions have an incompatible
prototype for idle() in <unistd.h> (which i thought was against namespace
and sensibility guidelines, but...)
* consistently use xsignal() instead of signal(). we get known behaviour
in all cases (SA_RESTART), which is good for some borken foreign systems.
* remove signal.h from most files; it's unnecessary now
* fetch_url(): use `long chunksize' instead of ssize_t; it's more portable, and
we're setting chunksize with strtol() anyway
* xsignal(): only use SA_RESTART if it exists. SunOS 4.x doesn't have it
but has the inverse (SA_INTERRUPT). the original function i was inspired
from had this support (lib/signal.c, W. Richard Stevens' `UNP 2nd ed Vol 1').
* remove <termios.h> from util.c; it should be unnecessary now
 1.23 22-Sep-1999  lukem branches: 1.23.2;
replace snprintf() with strlcpy(), strlcat(), or direct assignment
where appropriate. (strlc*() are easier to port to foriegn systems).

XXX; there's still a few snprintf's in the progress meter stuff to convert
 1.22 24-Jun-1999  christos include <signal.h> explicitly! we use sig_t extern.h, and we cannot
depend on other headers fetching it on other OS's.
 1.21 08-Aug-1998  lukem include <netbsd.h> for MAXHOSTNAMELEN on borken systems
 1.20 03-Aug-1998  lukem features:
* support $no_proxy, which is a comma or space separated list of
host[:port] elements for which proxying is to be disabled.
(asked for by cgd in [bin/5027])
* if $FTPANONPASS is defined, use that as the anon ftp password
(instead of "`whoami`@")
* allow http URL's without a filename as long as an output file
is specified.

other stuff:
* implement parse_url(), which breaks up a URL into its bits, and use.
* simplify url_get() and auto_fetch() to use parse_url() and to not
modify the supplied URL or a copy of it.
* implement xmalloc() and xstrdup(); error-checked malloc()/strdup()
* add more consistency to messages, quoting strings in output as `%s'
 1.19 26-Jul-1998  mycroft const poisoning.
 1.18 06-Jul-1998  mrg - use an array MAXHOSTNAMELEN+1 size to hold hostnames
- ensure hostname from gethostname() is nul-terminated in all cases
- minor KNF
- use MAXHOSTNAMELEN over various other values/defines
- be safe will buffers that hold hostnames
 1.17 04-Jun-1998  lukem some fixes & enhancements from openbsd's ftp, with extra fixes by me:
* default to passive with active fallback. $FTPMODE modifies this behaviour.
-A forces active connection.
* support '-o outfile' for auto-fetched files. outfile can be a file,
`-' (for stdout), or '|command' (to output each file through command).
* support '-r waittime', which retries the connection after waittime seconds
if it fails.
* fix 'page file' when restart is non-zero.
* try all ip-addresses of a host in a http fetch (as the normal ftp fetch
does).

XXX: a ``broken pipe'' error sometimes occurs with -o '|command';
i haven't tracked this down yet.
 1.16 20-May-1998  christos - clarify dangling else
 1.15 01-Nov-1997  lukem * in recvrequest(), ignore restart_point unless "RETR"ieving. fixes problems
where a remote completion or `mget' would confuse the client a `restart'
had been issued beforehand. now, `restart' is remembered until an operation
that can actually use it is invoked.
* in sendrequest(), don't reset restart_point upon entry. fixes `restart'
for `put' operations.
* if `restart' is invoked with no arguments, print current setting instead
of displaying a usage
* consistently use printf("%qd", (long long)restart_point) when displaying
restart_point
* use strto[lq]() instead of atol() when parsing `mark' and `restart' values
* remove unnecessary strlen()s when result of previous snprintf() will do
* replace a few malloc()/strcpy()s with strdup()s
* use SECSPERHOUR instead of '3600'
 1.14 20-Jul-1997  lukem branches: 1.14.2;
* use RCSID() && COPYRIGHT() macros
* cleanup for WARNS=1 (including some ugly '(void)&var' bits wrapped in
#ifdef __GNUC__ to shut up gcc warnings WRT setjmp/longjmp)
* use strtol() instead of atoi(), and more extensively check result of
conversion
* use u_int16_t instead of short or int for TCP port addresses
 1.13 01-Apr-1997  mrg fix buffer overrun; from bostic (via perry (via freebsd))
 1.12 13-Mar-1997  lukem Features:
* support remglobbing of auto_fetch arguments
* new flag - '-e'; disable editing
* "page file" == "get file |${PAGER-less}"

Bugfixes/cleanup:
* consistently use a trailing '.' on messages
* code cleanup, including buffer overrun fixes, use puts
and putchar in places, etc (inspired by OpenBSD mods)
* disable progress bar when local-file is a pipe or '-'
* skip \r in http headers
* fix remote ftpd slash bug more elegantly (so it works with ////)
* abort_remote(): check if cout==NULL before using it. should fix [bin/3273]
* fixed up cosmetic problems when complete_remote() generated errors from the
remote server (such as "no files found", "login with user and pass", ...)
done by adding extra argument to remglob(), which is a pointer to an error
buffer to put messages in rather than printing to stdout.
 1.11 19-Jan-1997  lukem New features:
* Command line editing via editline(3) library.
* Context sensitive command and file completion, including remote files.

Enhancements to auto-fetch feature:
* Support for http:// URLs using the http protocol, including proxy HTTP
support via $htty_proxy if it's defined.
* The connection is kept open between successive files on the same host.
(obviously, this does not count for http requests.)
* Return value of ftp is 0 on no error, or the offset in argv[] of the file
which failed (i.e., argv[x] failed, ftp returns x).
* If the path in an ftp URL or classic format line has a trailing '/',
cd to the path and enter interactive mode. Fixes [bin/3011], albiet
requiring the user to help ftp in determining the operation.

Other changes:
* '-P port' works for normal ftp, and is the default for all classic style
auto-fetch transfers and for ftp URLs that don't specify the port.
(previously it would just work for the first xfer.)
* Some code moved into separated files along logical divisions.
* Editing and completion can be compiled out with -DSMALLFTP.
 1.10 09-Jan-1997  tls RCS ID police
 1.9 06-Dec-1996  lukem functionality mods:
* implement 'progress bar/meter' (inspired by ncftp). use 'progress' to
toggle on. it will display current file size to 5 digits, automatically
determining suffix (up to 16384 P (petabytes) == 2^64).
* 'ls' now uses NLST (unadorned listing), a la older ftp clients. 'dir'
still does LIST (long listing). idea from John Nemeth <jnemeth@cue.bc.ca>

bug fixes:
* return first line of reply in reply_string[] from getreply(), instead
of last line. This fixes [bin/741] (parsing of SYST), and also means
that SIZE and MDTM messages will be parsed correctly if they're longer
than 1 line.
* parse URL-style auto-ftps that have no filename correctly
(e.g, ftp://host, ftp://host/, ftp://host/dir/). pointed out by
Jaromir Dolecek <dolecek@saruman.ics.muni.cz>
* pass the correct size array in 2nd arg of utimes() when setting the
modification time
 1.8 28-Nov-1996  lukem More features, some of which were inspired by changes that
friedman@gnu.ai.mit.edu (Noah Friedman) made to his modified ftp:
- implement "lpwd" - local pwd
- implement "preserve" - toggle preserving of file modification
times on retrieved files
- allow for explicit "on" or "off" arg to toggle commands
- "exit" synonym for "quit", "msend" synonym for "mput"
- in confirmation mode, allow 'a' (yes to rest of current command),
and 'p' (turn off prompt mode, as if 'prompt off' was done,
effective immediately)
- "modtime" returns time formatted as localtime, not GMT

Bug fixes:
- check for extraneous args on commands
- cleanup const usage, line formatting
- create 0 length temporary file in remglob() to prevent symlink games
(from OpenBSD)
- check length of filename of ~/.netrc (from OpenBSD)
 1.7 25-Nov-1996  lukem cleanups/bugfixes:
- don't echo 'ACCT' parameter when debugging (a la 'PASS')
- Fix checking of directory access for "/foo", the parent
directory is "/", not "" (from FreeBSD)
- remove trailing whitespace on lines
- add any missing NetBSD tags
- cleanups to man page, includinging sorting options description

feature additions:
- variable sized hash marks (from [bin/683], but done in the hash command
as an optional arg)
- more user-friendly transfer time printing (from FreeBSD, with mods)
- '-p' command line option to jump into PASV mode (closes [bin/2857],
but with an option rather than checking argv[0])
- SIGINFO support for printing xfer stats when sending/receiving requests
- '-P port' for changing the port to connect to (from thorpej@netbsd.org)
- '-a': bypass normal login, and try anonymous login (from OpenBSD
via thorpej)
- autofetch files via url (ftp://...) or "classic" (host:/file)
(from OpenBSD via thorpej)
- 'ftp' synonymous with 'open' (from FreeBSD)
 1.6 08-Sep-1995  tls Sync with 4.4lite2
 1.5 17-Sep-1994  mycroft Fix null pointer dereference.
 1.4 29-Aug-1994  mycroft branches: 1.4.2;
Add RCS ids.
 1.3 25-Aug-1994  cgd clean up import, no local changes.
 1.2 01-Aug-1993  mycroft Add RCS identifiers.
 1.1 21-Mar-1993  cgd branches: 1.1.1; 1.1.2;
Initial revision
 1.1.2.1 25-Aug-1994  cgd import from 4.4-Lite
 1.1.1.2 05-Sep-1995  tls imported from 44lite2
 1.1.1.1 21-Mar-1993  cgd initial import of 386bsd-0.1 sources
 1.4.2.2 29-Aug-1994  mycroft Add RCS ids.
 1.4.2.1 29-Aug-1994  mycroft file ruserpass.c was added on branch netbsd-1-0 on 1994-08-29 03:09:22 +0000
 1.14.2.2 10-Nov-1998  cgd pull up rev(s) 1.16-1.21 from trunk. (feyrer)
 1.14.2.1 18-Nov-1997  mellon Pull rev 1.15 up from trunk (lukem)
 1.23.2.1 27-Dec-1999  wrstuden Pull up to last week's -current.
 1.29.6.1 24-Jul-2005  tron Pull up revision 1.30 (requested by lukem in ticket #606):
Implement:
int getline(FILE *stream, char *buf, size_t buflen, const char **errormsg)
Read a line from the FILE stream into buf/buflen using fgets(), so up
to buflen-1 chars will be read and the result will be NUL terminated.
If the line has a trailing newline it will be removed.
If the line is too long, excess characters will be read until
newline/EOF/error.
Various -ve return values indicate different errors, and errormsg
will be changed to an error description if it's not NULL.
Convert to use getline() instead of fgets() whenever reading user input
to ensure that an overly long input line doesn't leave excess characters
for the next input operation to accidentally use as input.
Zero out the password & account after we've finished with it.
Consistently use getpass(3) (i.e, character echo suppressed) when
reading the account data. For some reason, historically the "login"
code suppressed echo for Account: yet the "user" command did not!
Display the hostname in the "getaddrinfo failed" warning.
Appease some -Wcast-qual warnings. Fixing all of these requires
significant code refactoring. (mmm, legacy code).
 1.33.88.1 02-Aug-2025  perseant Sync with HEAD
 1.33.86.2 02-Dec-2024  martin Pull up following revision(s) (requested by lukem in ticket #1021):

usr.bin/ftp/ftp.c: revision 1.178
usr.bin/ftp/version.h: revision 1.99
usr.bin/ftp/ruserpass.c: revision 1.35
usr.bin/ftp/main.c: revision 1.134
usr.bin/ftp/main.c: revision 1.135
usr.bin/ftp/progressbar.c: revision 1.27
usr.bin/ftp/util.c: revision 1.165
usr.bin/ftp/cmds.c: revision 1.144
usr.bin/ftp/extern.h: revision 1.84
usr.bin/ftp/fetch.c: revision 1.242
usr.bin/ftp/ftp.1: revision 1.160

s/bninary/binary/ in comment.

extract duplicate code into a function.

Check bounds when copying to destination.

Remove const where the const string ended up being overwritten.

use unsigned when doing shifts.

remove const when string gets overwritten.

ftp: exit non-zero if short http transfer when filesize is known
If a http file size is known and the fetch finishes with less bytes
transferred, exit non-zero.
Bump version to 20241129.
PR bin/54713
PR bin/58281

ftp: help improvements
Document -? as a separate mode.
Document -H HEADER in the usage.
Clarify units for -b and -x.
Consistent argument names between ftp -? and ftp(1).

ftp: order getopt Upper before lower
Consistently order options in getopt and the switch
with the upper case option before the lower case option.
This makes it easier to cross-reference with -? and ftp(1).
No functional change.
 1.33.86.1 13-Oct-2024  martin Pull up following revision(s) (requested by riastradh in ticket #970):

tests/usr.bin/Makefile: revision 1.42
usr.bin/ftp/ruserpass.c: revision 1.34
usr.bin/ftp/main.c: revision 1.130
usr.bin/ftp/ssl.c: revision 1.17
usr.bin/ftp/main.c: revision 1.131
usr.bin/ftp/ssl.c: revision 1.18
usr.bin/ftp/main.c: revision 1.132
usr.bin/ftp/ssl.c: revision 1.19
usr.bin/ftp/main.c: revision 1.133
distrib/sets/lists/tests/mi: revision 1.1342
usr.bin/ftp/ftp.1: revision 1.151
usr.bin/ftp/ftp.1: revision 1.152
usr.bin/ftp/progressbar.c: revision 1.25
usr.bin/ftp/ftp.1: revision 1.153
usr.bin/ftp/progressbar.c: revision 1.26
usr.bin/ftp/ftp.1: revision 1.155
usr.bin/ftp/ftp.1: revision 1.156
usr.bin/ftp/fetch.c: revision 1.239
usr.bin/ftp/ftp.1: revision 1.157
usr.bin/ftp/ftp.1: revision 1.158
usr.bin/ftp/ftp.1: revision 1.159
usr.bin/ftp/ftp_var.h: revision 1.87
etc/mtree/NetBSD.dist.tests: revision 1.208
usr.bin/ftp/ftp_var.h: revision 1.88
usr.bin/ftp/ftp_var.h: revision 1.89
usr.bin/ftp/cmds.c: revision 1.142
usr.bin/ftp/util.c: revision 1.168
usr.bin/ftp/cmds.c: revision 1.143
tests/usr.bin/ftp/custom_headers.sh: revision 1.1
usr.bin/ftp/ssl.c: revision 1.20
usr.bin/ftp/complete.c: revision 1.48
tests/usr.bin/ftp/Makefile: revision 1.1
tests/usr.bin/ftp/t_custom_headers.sh: revision 1.1
usr.bin/ftp/fetch.c: revision 1.240
usr.bin/ftp/fetch.c: revision 1.241
usr.bin/ftp/ftp.c: revision 1.176
usr.bin/ftp/ftp.c: revision 1.177
(all via patch)

ftp(1): wording and formatting improvements

Fix grammar issue with "Support values" reported in private mail.
Document all file transfer types in "type" and cross-reference that.
Consistency fixes in describing file transfer parameters and types.

Fix some mandoc -Tlint issues (except "useless macro: Tn").

Add -b <buflen> to specify the buffer size.

ftp: bump FTPBUFLEN from 4kB to 16kB
sourceforge.net returns a 5kB content-security-policy.
Analyzed by mlelstv@ who reports usual limits are between 4kB and 48kB.
default is now 16K

ftp: improve -b documentation

Order -b bufsize in the synopsis.

Document the actual default value.

ftp: improve units used in comments and errors
Use "KiB" instead of "K" in errors.
Clarify related comments.

pass some lint.

PR/58581: Sunil Nimmagadda: Add flag to allow specifying extra http header
fields.

ftp(1): Nix trailing whitespace in man page.
No functional change intended.

PR bin/58581: ftp(1) should allow specifying header fields in http requests
fix markup (h -> H), explain about multiple headers, fix usage (from RVP)

Don't forget the dot, use the intended macro name (I think),
and improve the wording a little. (All related to the -H option.)
ftp(1): Add test for custom HTTP header fields.

Based on a patch from Sunil Nimmagadda.

PR bin/58581: ftp(1) should allow specifying header fields in http
requests
 1.20 25-Sep-2024  christos pass some lint.
 1.19 19-Jul-2024  lukem ftp: improve -b documentation

Order -b bufsize in the synopsis.
Document the actual default value.
 1.18 19-Feb-2024  christos branches: 1.18.2;
default is now 16K
 1.17 18-Feb-2024  christos Add -b <buflen> to specify the buffer size.
 1.16 16-May-2023  christos Ignore EOF from remote.
 1.15 05-May-2023  lukem add timeout for ssl connect

Implement a timeout for SSL connection setup, using -q QUITTIME,
defaulting to 60 seconds.
SSL_connect(3) (unlike connect(2)) doesn't timeout by default.

Adapt ssl error messages destination: if unexpected error
from local API, use warn()/warnx() to stderr;
if expected error from a network operation (e.g., timeouts),
use fprintf to ttyout (which might be stdout).

Consistently use ftp_poll() instead of select();
ssl.c (using select()) was added 7 years after the
previous uses of select() were converted to poll().

Check EAGAIN as well as existing EINTR error from ftp_poll(),
for portability.
 1.14 09-Apr-2023  lukem Simplify includes

Include "ftp_var.h" instead of various system headers and "extern.h".
 1.13 25-Feb-2023  mlelstv Add option sslnoverify to control validation of SSL certificates.
Add netrc processing to fetch-mode (URL on command line) to enable options and autologin
via netrc.
Fix SSL cleanup in some error paths.

Certificate validation is now enabled by default. Set FTPSSLNOVERIFY=1 in environment
or configure a corresponding init macro via netrc to not validate certs (required if
you haven't installed a required CA certificate for OpenSSL).

Discussed with lukem@ on icb.
 1.12 12-Sep-2022  christos branches: 1.12.2;
Disable verification for now until we implement installation of trust anchors.
 1.11 30-Aug-2022  christos Add cert verification, together with an environment variable "NO_CERT_VERIFY",
to turn it off.
 1.10 03-Jun-2021  lukem use fetch_*() for I/O with SMALLPROG / !WITH_SSL builds

Adapt the SMALLPROG / -UWITH_SSL build to also use the fetch_*()
methods from ssl.c, instead of using stdio, as stdio isn't robust
when using interruptable signals.

Disable ssl-specific support in the fetch_*() methods if WITH_SSL
isn't defined, so SMALLPROG still doesn't have ssl support (as expected).

The resulting SMALLPROG binary is slightly larger than before
(e.g., 157KiB vs 153KiB on amd64).

Set version to 20210603 for this fix and the SO_KEEPALIVE fix for PR 56129.

PR install/56219
 1.9 06-Jan-2021  lukem branches: 1.9.4;
ftp: don't use restartable signals

Refactor to not rely upon restartable signals (SA_RESTART),
possibly fixing intermittent failures with -q QUITTIME.

ftp transfers: handle EINTR/EAGAIN in copy_bytes(),
instead of relying upon restartable signals.

http/https transfers: Explicitly print an error similar to
progressmeter() when timing-out for -Q QUITTIME in fetch_wait(),
and set errno to ETIMEDOUT so that the warn() in fetch_url()
prints a more accurate error message.

PR/55857
 1.8 07-Apr-2019  christos branches: 1.8.2;
redo the connection waiting handling to make it more clear.
 1.7 04-Apr-2019  christos Make fetch_read() return size_t like fread() does. It is bogus to
have one backing implementation that returns different values and
types than the other. Handle error setting properly; i.e. bail
out if the internal read returned an error. Now we get a proper
error message when the the server resets our connection instead of
a warning that the right failed with an invalid argument.

The server used for testing was:
http://capeweather.dyndns.org:8080/graphs/3474.png
Which seems to be unreliable :-)
 1.6 06-Feb-2018  christos branches: 1.6.4;
explicitly include <string.h> since OpenSSL-1.1 does not do it for us.
 1.5 16-Sep-2015  joerg branches: 1.5.8;
Workaround const issues of SSL_set_tlsext_host_name.
 1.4 12-Sep-2015  wiz servername cannot be NULL here.

Noted by joerg@.
 1.3 12-Sep-2015  wiz Add Server Name Indication (SNI) support for https.

Needed for e.g. some github URLs.
 1.2 24-Dec-2012  christos branches: 1.2.2; 1.2.6; 1.2.10; 1.2.14;
backwards compatible with netbsd-6.
 1.1 21-Dec-2012  christos PR/47276: Add https support
 1.2.14.2 13-Mar-2016  martin Pull up following revision(s) (requested by nonakap in ticket #1133):
usr.bin/ftp/fetch.c: revision 1.208-1.221
usr.bin/ftp/cmds.c: revision 1.136-1.137
usr.bin/ftp/ssl.c: revision 1.5
usr.bin/ftp/ftp.c: revision 1.165-1.166
usr.bin/ftp/ftp_var.h: revision 1.84

Workaround const issues of SSL_set_tlsext_host_name.

Use the proper format "[IPv6 address]:port" when reporting connection
attempts to IPv6 endpoints.

(Hopefully) fix build without IPv6 support

Try to factor out some code, this is completely out of control.

Separate no_proxy handling.

Factor the proxy handling code out.

Fix compile failure without WITH_SSL.

PR/50438: NONAKA Kimihiro: ftp(1): CONNECT method support

make DPRINTF/DWARN always statements.

Fix to connect https via proxy.

Fix ttyout message.

Simplify and factor out connect message

Split the position/size parsing into a separate function.

Mark function as only needed with ssl.

Fix downloads of local files using file:// URLs

Initialize the token match pointer.

use sizeof() and array notation.
CID 1354295: Array overrun.
 1.2.14.1 05-Nov-2015  riz Pull up following revision(s) (requested by wiz in ticket #981):
usr.bin/ftp/ftp.1: revision 1.135
usr.bin/ftp/ssl.c: revision 1.3
usr.bin/ftp/ssl.c: revision 1.4
usr.bin/ftp/ssl.h: revision 1.3
usr.bin/ftp/version.h: revision 1.86
usr.bin/ftp/version.h: revision 1.87
usr.bin/ftp/fetch.c: revision 1.207
usr.bin/ftp/main.c: revision 1.123
Add -x xferbufsize to set xferbuf size.
Implement -x xferbufsize set the socket send and receive buffer size,
as per 'xferbuf' in interactive mode.
Patch from Nicholas Mills (via private mail), with minor adjustment by me.
Add Server Name Indication (SNI) support for https.
Needed for e.g. some github URLs.
Bump version for SNI support.
servername cannot be NULL here.
Noted by joerg@.
 1.2.10.3 27-Aug-2016  bouyer Apply patch, requested by nonaka in ticket #1375:
src/usr.bin/ftp/cmds.c: patch
src/usr.bin/ftp/fetch.c: patch
src/usr.bin/ftp/ftp.1: patch
src/usr.bin/ftp/ftp.c: patch
src/usr.bin/ftp/ftp_var.h: patch
src/usr.bin/ftp/main.c: patch
src/usr.bin/ftp/ssl.c: patch
src/usr.bin/ftp/ssl.h: patch
src/usr.bin/ftp/version.h: patch
Update ftp(1) to version 20150912, adding https via proxy support.
 1.2.10.2 17-Dec-2013  bouyer Apply patch, requested by tron in ticket #997:
usr.bin/ftp/Makefile patch
usr.bin/ftp/cmds.c patch
usr.bin/ftp/cmdtab.c patch
usr.bin/ftp/extern.h patch
usr.bin/ftp/fetch.c patch
usr.bin/ftp/ftp.1 patch
usr.bin/ftp/ftp.c patch
usr.bin/ftp/ftp_var.h patch
usr.bin/ftp/main.c patch
usr.bin/ftp/progressbar.c patch
usr.bin/ftp/ssl.c patch
usr.bin/ftp/ssl.h patch
usr.bin/ftp/util.c patch
usr.bin/ftp/version.h patch

Add HTTPS support to ftp(1).
 1.2.10.1 24-Dec-2012  bouyer file ssl.c was added on branch netbsd-6 on 2013-12-17 21:07:59 +0000
 1.2.6.2 25-Feb-2013  tls resync with head
 1.2.6.1 24-Dec-2012  tls file ssl.c was added on branch tls-maxphys on 2013-02-25 00:30:35 +0000
 1.2.2.2 23-Jan-2013  yamt sync with head
 1.2.2.1 24-Dec-2012  yamt file ssl.c was added on branch yamt-pagecache on 2013-01-23 00:06:38 +0000
 1.5.8.3 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1763:

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/cmds.c up to 1.141
usr.bin/ftp/complete.c up to 1.47
usr.bin/ftp/domacro.c up to 1.23
usr.bin/ftp/extern.h up to 1.82
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp.c up to 1.174
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/progressbar.c up to 1.24
usr.bin/ftp/progressbar.h up to 1.9
usr.bin/ftp/ssl.c up to 1.12
usr.bin/ftp/ssl.h up to 1.5
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

ftp(1): validate address from PASV and LPSV response.
ftp(1): use raw write(2) instead of fwrite(3) to avoid stream
corruption because of the progress bar interrupts.
Fixes for PR 56219 and PR 55857.
PR 57003: Support relative redirects.
 1.5.8.2 12-Sep-2022  martin Backout ticket #1763 for now - trust anchors are not solved.
 1.5.8.1 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1763:

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/cmds.c up to 1.141
usr.bin/ftp/complete.c up to 1.47
usr.bin/ftp/domacro.c up to 1.23
usr.bin/ftp/extern.h up to 1.82
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp.c up to 1.174
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/progressbar.c up to 1.24
usr.bin/ftp/progressbar.h up to 1.9
usr.bin/ftp/ssl.c up to 1.11
usr.bin/ftp/ssl.h up to 1.5
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

ftp(1): validate address from PASV and LPSV response.
ftp(1): use raw write(2) instead of fwrite(3) to avoid stream
corruption because of the progress bar interrupts.
Fixes for PR 56219 and PR 55857.
PR 57003: Support relative redirects.
 1.6.4.1 10-Jun-2019  christos Sync with HEAD
 1.8.2.5 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1523

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/ssl.c up to 1.12
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

PR 57003: Support relative redirects.
 1.8.2.4 12-Sep-2022  martin Back out ticket #1523 for now - trust anchor validation is not solved.
 1.8.2.3 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1523

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/ssl.c up to 1.11
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

PR 57003: Support relative redirects.
 1.8.2.2 14-Jun-2021  martin Pull up following revision(s) (requested by lukem in ticket #1295):

usr.bin/ftp/ssl.c: revision 1.10
usr.bin/ftp/ssl.h: revision 1.5
usr.bin/ftp/version.h: revision 1.93
usr.bin/ftp/Makefile: revision 1.39

use fetch_*() for I/O with SMALLPROG / !WITH_SSL builds

Adapt the SMALLPROG / -UWITH_SSL build to also use the fetch_*()
methods from ssl.c, instead of using stdio, as stdio isn't robust
when using interruptable signals.

Disable ssl-specific support in the fetch_*() methods if WITH_SSL
isn't defined, so SMALLPROG still doesn't have ssl support (as expected).

The resulting SMALLPROG binary is slightly larger than before
(e.g., 157KiB vs 153KiB on amd64).

Set version to 20210603 for this fix and the SO_KEEPALIVE fix for PR 56129.

PR install/56219
 1.8.2.1 29-Jan-2021  martin Pull up following revision(s) (requested by lukem in ticket #1190):

usr.bin/ftp/progressbar.c: revision 1.24
usr.bin/ftp/ssl.c: revision 1.9
usr.bin/ftp/progressbar.h: revision 1.9
usr.bin/ftp/ftp.c: revision 1.171
usr.bin/ftp/version.h: revision 1.92

ftp: don't use restartable signals

Refactor to not rely upon restartable signals (SA_RESTART),
possibly fixing intermittent failures with -q QUITTIME.
ftp transfers: handle EINTR/EAGAIN in copy_bytes(),
instead of relying upon restartable signals.

http/https transfers: Explicitly print an error similar to
progressmeter() when timing-out for -Q QUITTIME in fetch_wait(),
and set errno to ETIMEDOUT so that the warn() in fetch_url()
prints a more accurate error message.

PR/55857
 1.9.4.1 06-Jun-2021  cjep sync with head
 1.12.2.5 13-Oct-2024  martin Pull up following revision(s) (requested by riastradh in ticket #970):

tests/usr.bin/Makefile: revision 1.42
usr.bin/ftp/ruserpass.c: revision 1.34
usr.bin/ftp/main.c: revision 1.130
usr.bin/ftp/ssl.c: revision 1.17
usr.bin/ftp/main.c: revision 1.131
usr.bin/ftp/ssl.c: revision 1.18
usr.bin/ftp/main.c: revision 1.132
usr.bin/ftp/ssl.c: revision 1.19
usr.bin/ftp/main.c: revision 1.133
distrib/sets/lists/tests/mi: revision 1.1342
usr.bin/ftp/ftp.1: revision 1.151
usr.bin/ftp/ftp.1: revision 1.152
usr.bin/ftp/progressbar.c: revision 1.25
usr.bin/ftp/ftp.1: revision 1.153
usr.bin/ftp/progressbar.c: revision 1.26
usr.bin/ftp/ftp.1: revision 1.155
usr.bin/ftp/ftp.1: revision 1.156
usr.bin/ftp/fetch.c: revision 1.239
usr.bin/ftp/ftp.1: revision 1.157
usr.bin/ftp/ftp.1: revision 1.158
usr.bin/ftp/ftp.1: revision 1.159
usr.bin/ftp/ftp_var.h: revision 1.87
etc/mtree/NetBSD.dist.tests: revision 1.208
usr.bin/ftp/ftp_var.h: revision 1.88
usr.bin/ftp/ftp_var.h: revision 1.89
usr.bin/ftp/cmds.c: revision 1.142
usr.bin/ftp/util.c: revision 1.168
usr.bin/ftp/cmds.c: revision 1.143
tests/usr.bin/ftp/custom_headers.sh: revision 1.1
usr.bin/ftp/ssl.c: revision 1.20
usr.bin/ftp/complete.c: revision 1.48
tests/usr.bin/ftp/Makefile: revision 1.1
tests/usr.bin/ftp/t_custom_headers.sh: revision 1.1
usr.bin/ftp/fetch.c: revision 1.240
usr.bin/ftp/fetch.c: revision 1.241
usr.bin/ftp/ftp.c: revision 1.176
usr.bin/ftp/ftp.c: revision 1.177
(all via patch)

ftp(1): wording and formatting improvements

Fix grammar issue with "Support values" reported in private mail.
Document all file transfer types in "type" and cross-reference that.
Consistency fixes in describing file transfer parameters and types.

Fix some mandoc -Tlint issues (except "useless macro: Tn").

Add -b <buflen> to specify the buffer size.

ftp: bump FTPBUFLEN from 4kB to 16kB
sourceforge.net returns a 5kB content-security-policy.
Analyzed by mlelstv@ who reports usual limits are between 4kB and 48kB.
default is now 16K

ftp: improve -b documentation

Order -b bufsize in the synopsis.

Document the actual default value.

ftp: improve units used in comments and errors
Use "KiB" instead of "K" in errors.
Clarify related comments.

pass some lint.

PR/58581: Sunil Nimmagadda: Add flag to allow specifying extra http header
fields.

ftp(1): Nix trailing whitespace in man page.
No functional change intended.

PR bin/58581: ftp(1) should allow specifying header fields in http requests
fix markup (h -> H), explain about multiple headers, fix usage (from RVP)

Don't forget the dot, use the intended macro name (I think),
and improve the wording a little. (All related to the -H option.)
ftp(1): Add test for custom HTTP header fields.

Based on a patch from Sunil Nimmagadda.

PR bin/58581: ftp(1) should allow specifying header fields in http
requests
 1.12.2.4 26-Aug-2024  martin Pull up following revision(s) (requested by wiz in ticket #814):

usr.bin/ftp/ssl.c: revision 1.16

Ignore EOF from remote.
 1.12.2.3 16-May-2023  martin Pull up following revision(s) (requested by lukem in ticket #171):

usr.bin/ftp/ssl.c: revision 1.15
usr.bin/ftp/util.c: revision 1.167
usr.bin/ftp/ftp.c: revision 1.175
usr.bin/ftp/version.h: revision 1.97

add timeout for ssl connect

Implement a timeout for SSL connection setup, using -q QUITTIME,
defaulting to 60 seconds.

SSL_connect(3) (unlike connect(2)) doesn't timeout by default.
Adapt ssl error messages destination: if unexpected error
from local API, use warn()/warnx() to stderr;
if expected error from a network operation (e.g., timeouts),
use fprintf to ttyout (which might be stdout).

Consistently use ftp_poll() instead of select();
ssl.c (using select()) was added 7 years after the
previous uses of select() were converted to poll().

Check EAGAIN as well as existing EINTR error from ftp_poll(),
for portability.
 1.12.2.2 16-May-2023  martin Pull up following revision(s) (requested by lukem in ticket #174):

usr.bin/ftp/ssl.c: revision 1.14

Simplify includes

Include "ftp_var.h" instead of various system headers and "extern.h".
 1.12.2.1 16-May-2023  martin Pull up following revision(s) (requested by lukem in ticket #170):

usr.bin/ftp/ssl.c: revision 1.13
usr.bin/ftp/fetch.c: revision 1.236
usr.bin/ftp/util.c: revision 1.166
usr.bin/ftp/main.c: revision 1.129
usr.bin/ftp/extern.h: revision 1.83
usr.bin/ftp/ftp.1: revision 1.148
usr.bin/ftp/cmdtab.c: revision 1.53
usr.bin/ftp/version.h: revision 1.96

Add option sslnoverify to control validation of SSL certificates.

Add netrc processing to fetch-mode (URL on command line) to enable options and autologin
via netrc.

Fix SSL cleanup in some error paths.

Certificate validation is now enabled by default. Set FTPSSLNOVERIFY=1 in environment
or configure a corresponding init macro via netrc to not validate certs (required if
you haven't installed a required CA certificate for OpenSSL).

Discussed with lukem@ on icb.
 1.18.2.1 02-Aug-2025  perseant Sync with HEAD
 1.5 03-Jun-2021  lukem use fetch_*() for I/O with SMALLPROG / !WITH_SSL builds

Adapt the SMALLPROG / -UWITH_SSL build to also use the fetch_*()
methods from ssl.c, instead of using stdio, as stdio isn't robust
when using interruptable signals.

Disable ssl-specific support in the fetch_*() methods if WITH_SSL
isn't defined, so SMALLPROG still doesn't have ssl support (as expected).

The resulting SMALLPROG binary is slightly larger than before
(e.g., 157KiB vs 153KiB on amd64).

Set version to 20210603 for this fix and the SO_KEEPALIVE fix for PR 56129.

PR install/56219
 1.4 04-Apr-2019  christos branches: 1.4.2; 1.4.8;
Make fetch_read() return size_t like fread() does. It is bogus to
have one backing implementation that returns different values and
types than the other. Handle error setting properly; i.e. bail
out if the internal read returned an error. Now we get a proper
error message when the the server resets our connection instead of
a warning that the right failed with an invalid argument.

The server used for testing was:
http://capeweather.dyndns.org:8080/graphs/3474.png
Which seems to be unreliable :-)
 1.3 12-Sep-2015  wiz branches: 1.3.8; 1.3.16;
Add Server Name Indication (SNI) support for https.

Needed for e.g. some github URLs.
 1.2 07-Jan-2014  joerg branches: 1.2.4;
Annotate functions using format strings.
 1.1 21-Dec-2012  christos branches: 1.1.2; 1.1.6; 1.1.10;
PR/47276: Add https support
 1.1.10.3 27-Aug-2016  bouyer Apply patch, requested by nonaka in ticket #1375:
src/usr.bin/ftp/cmds.c: patch
src/usr.bin/ftp/fetch.c: patch
src/usr.bin/ftp/ftp.1: patch
src/usr.bin/ftp/ftp.c: patch
src/usr.bin/ftp/ftp_var.h: patch
src/usr.bin/ftp/main.c: patch
src/usr.bin/ftp/ssl.c: patch
src/usr.bin/ftp/ssl.h: patch
src/usr.bin/ftp/version.h: patch
Update ftp(1) to version 20150912, adding https via proxy support.
 1.1.10.2 17-Dec-2013  bouyer Apply patch, requested by tron in ticket #997:
usr.bin/ftp/Makefile patch
usr.bin/ftp/cmds.c patch
usr.bin/ftp/cmdtab.c patch
usr.bin/ftp/extern.h patch
usr.bin/ftp/fetch.c patch
usr.bin/ftp/ftp.1 patch
usr.bin/ftp/ftp.c patch
usr.bin/ftp/ftp_var.h patch
usr.bin/ftp/main.c patch
usr.bin/ftp/progressbar.c patch
usr.bin/ftp/ssl.c patch
usr.bin/ftp/ssl.h patch
usr.bin/ftp/util.c patch
usr.bin/ftp/version.h patch

Add HTTPS support to ftp(1).
 1.1.10.1 21-Dec-2012  bouyer file ssl.h was added on branch netbsd-6 on 2013-12-17 21:07:59 +0000
 1.1.6.3 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.6.2 25-Feb-2013  tls resync with head
 1.1.6.1 21-Dec-2012  tls file ssl.h was added on branch tls-maxphys on 2013-02-25 00:30:35 +0000
 1.1.2.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.2.2 23-Jan-2013  yamt sync with head
 1.1.2.1 21-Dec-2012  yamt file ssl.h was added on branch yamt-pagecache on 2013-01-23 00:06:38 +0000
 1.2.4.1 05-Nov-2015  riz Pull up following revision(s) (requested by wiz in ticket #981):
usr.bin/ftp/ftp.1: revision 1.135
usr.bin/ftp/ssl.c: revision 1.3
usr.bin/ftp/ssl.c: revision 1.4
usr.bin/ftp/ssl.h: revision 1.3
usr.bin/ftp/version.h: revision 1.86
usr.bin/ftp/version.h: revision 1.87
usr.bin/ftp/fetch.c: revision 1.207
usr.bin/ftp/main.c: revision 1.123
Add -x xferbufsize to set xferbuf size.
Implement -x xferbufsize set the socket send and receive buffer size,
as per 'xferbuf' in interactive mode.
Patch from Nicholas Mills (via private mail), with minor adjustment by me.
Add Server Name Indication (SNI) support for https.
Needed for e.g. some github URLs.
Bump version for SNI support.
servername cannot be NULL here.
Noted by joerg@.
 1.3.16.1 10-Jun-2019  christos Sync with HEAD
 1.3.8.3 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1763:

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/cmds.c up to 1.141
usr.bin/ftp/complete.c up to 1.47
usr.bin/ftp/domacro.c up to 1.23
usr.bin/ftp/extern.h up to 1.82
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp.c up to 1.174
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/progressbar.c up to 1.24
usr.bin/ftp/progressbar.h up to 1.9
usr.bin/ftp/ssl.c up to 1.12
usr.bin/ftp/ssl.h up to 1.5
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

ftp(1): validate address from PASV and LPSV response.
ftp(1): use raw write(2) instead of fwrite(3) to avoid stream
corruption because of the progress bar interrupts.
Fixes for PR 56219 and PR 55857.
PR 57003: Support relative redirects.
 1.3.8.2 12-Sep-2022  martin Backout ticket #1763 for now - trust anchors are not solved.
 1.3.8.1 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1763:

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/cmds.c up to 1.141
usr.bin/ftp/complete.c up to 1.47
usr.bin/ftp/domacro.c up to 1.23
usr.bin/ftp/extern.h up to 1.82
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp.c up to 1.174
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/progressbar.c up to 1.24
usr.bin/ftp/progressbar.h up to 1.9
usr.bin/ftp/ssl.c up to 1.11
usr.bin/ftp/ssl.h up to 1.5
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

ftp(1): validate address from PASV and LPSV response.
ftp(1): use raw write(2) instead of fwrite(3) to avoid stream
corruption because of the progress bar interrupts.
Fixes for PR 56219 and PR 55857.
PR 57003: Support relative redirects.
 1.4.8.1 06-Jun-2021  cjep sync with head
 1.4.2.2 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1523

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/ssl.c up to 1.12
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

PR 57003: Support relative redirects.
 1.4.2.1 14-Jun-2021  martin Pull up following revision(s) (requested by lukem in ticket #1295):

usr.bin/ftp/ssl.c: revision 1.10
usr.bin/ftp/ssl.h: revision 1.5
usr.bin/ftp/version.h: revision 1.93
usr.bin/ftp/Makefile: revision 1.39

use fetch_*() for I/O with SMALLPROG / !WITH_SSL builds

Adapt the SMALLPROG / -UWITH_SSL build to also use the fetch_*()
methods from ssl.c, instead of using stdio, as stdio isn't robust
when using interruptable signals.

Disable ssl-specific support in the fetch_*() methods if WITH_SSL
isn't defined, so SMALLPROG still doesn't have ssl support (as expected).

The resulting SMALLPROG binary is slightly larger than before
(e.g., 157KiB vs 153KiB on amd64).

Set version to 20210603 for this fix and the SO_KEEPALIVE fix for PR 56129.

PR install/56219
 1.168 25-Sep-2024  christos pass some lint.
 1.167 05-May-2023  lukem branches: 1.167.2;
add timeout for ssl connect

Implement a timeout for SSL connection setup, using -q QUITTIME,
defaulting to 60 seconds.
SSL_connect(3) (unlike connect(2)) doesn't timeout by default.

Adapt ssl error messages destination: if unexpected error
from local API, use warn()/warnx() to stderr;
if expected error from a network operation (e.g., timeouts),
use fprintf to ttyout (which might be stdout).

Consistently use ftp_poll() instead of select();
ssl.c (using select()) was added 7 years after the
previous uses of select() were converted to poll().

Check EAGAIN as well as existing EINTR error from ftp_poll(),
for portability.
 1.166 25-Feb-2023  mlelstv Add option sslnoverify to control validation of SSL certificates.
Add netrc processing to fetch-mode (URL on command line) to enable options and autologin
via netrc.
Fix SSL cleanup in some error paths.

Certificate validation is now enabled by default. Set FTPSSLNOVERIFY=1 in environment
or configure a corresponding init macro via netrc to not validate certs (required if
you haven't installed a required CA certificate for OpenSSL).

Discussed with lukem@ on icb.
 1.165 20-Jan-2023  andvar s/bninary/binary/ in comment.
 1.164 06-Aug-2022  andvar branches: 1.164.2;
s/blity/bility/ in various words, mainly in comments.
 1.163 27-Aug-2021  rillig ftp: fix undefined behavior when parsing FEAT

lint says: warning: argument to 'function from <ctype.h>' must be cast
to 'unsigned char', not to 'int' [342]
 1.162 25-Apr-2021  lukem better XXX comment for buggy ftp server
 1.161 08-Jun-2020  lukem ftp: exit if lostpeer invoked by a signal

lostpeer() calls too many async-unsafe functions (both directly
and indirectly) to close and cleanup the remote connections,
so just exit after the cleanup if invoked by a signal.

Reported in private mail by Qi Hou.
May also resolve a crash reported by Thomas Klausner.
 1.160 22-Jun-2019  christos branches: 1.160.2;
trim down error checking if we are small.
 1.159 20-Nov-2017  kre branches: 1.159.4;
Issue PWD commands to the server only when we actually
need the results, not speculatively, just in case we might.

Allows operation with some broken servers that get confused
by PWD commands in some situations, and saves server round
trips in the (modern) common case of
ftp ftp://path/name
where we never need to know the results from PWD.
 1.158 19-Feb-2013  dsl branches: 1.158.22;
When using the response to SYST to decide whether to default to 'binary'
be a lot less specific.
Kyocera printers report "230 Linux" but really don't want text transfers
of pdf files!
 1.157 04-Jul-2012  is branches: 1.157.2;
As discussed on tech-net@: Don't display expected EHOSTUNREACH for all but
the last connect attempts in terse mode.
 1.156 10-Dec-2011  lukem branches: 1.156.2;
Move determination of socket buffer sizes from startup to the first
time a socket is used, as the previous logic assumed AF_INET sockets
were available (which they may not be in an IPv6-only system).
Per discussion with Maxim Konovalov and the FreeBSD problem 162661.
 1.155 05-Jun-2010  lukem branches: 1.155.6;
In ftpvis(), prevent incomplete escape sequences at end of dst,
and ensure NUL-termination of dst. Also tweak for readibility.
Fix from Uwe Stuehler and Stefan Sperling, via Marc Balmer.
 1.154 05-Mar-2010  lukem Back to using 'RFC xxxx' instead of 'RFCxxxx'
 1.153 04-Mar-2010  lukem Parse HTTP 'Date' entries in the `C' locale rather than the user's.
Fix from [bin/42917] (with minor changes), from KAMADA Ken'ichi.
 1.152 13-Jul-2009  roy Rename internal getline() function to get_line() so it does
conflict with the soon to be added getline(3) libc function.
 1.151 15-Apr-2009  jld Another empty-body warning/error, another set of curly braces. This, and
the last one, appear to affect only the NO_DEBUG (e.g., SMALLPROG) build.
 1.150 12-Apr-2009  lukem Fix numerous WARNS=4 issues (-Wcast-qual -Wsign-compare).
 1.149 12-Apr-2009  lukem fix -Wshadow issues
 1.148 13-Aug-2008  lukem branches: 1.148.4; 1.148.6;
Use AF_INET instead of AF_UNSPEC as the default family if !defined(INET6).
Avoids problem reported by Matthias Scheler <tron@NetBSD.org>.
 1.147 10-May-2008  skd Add epsv6 and epsv to disable extended passive mode for ipv6 or both ipv4 and ipv6 respectively. This hack is due to our friends a Juniper Networks who break
epsv in ipv6. Should be fixed in ScreenOS 6.2.X.
 1.146 28-Apr-2008  martin branches: 1.146.2;
Remove clause 3 and 4 from TNF licenses
 1.145 22-Apr-2008  lukem Use the service name to getaddrinfo() (along with the host name), so that
features such as DNS Service Discovery have a better chance of working.
Suggested by David Young <dyoung>.

Display the service name in various status & error messages.

Don't getservbyname() the :port component of a URL; RFC 3986 says it's
just an unsigned number, not a service name.
 1.144 05-Dec-2007  lukem branches: 1.144.6;
DPRINTF() consistency tweaks
 1.143 24-May-2007  lukem branches: 1.143.4;
Display times in RFC2822 form rather than using ctime(3), since
the former is more explicit about the timezone offset.
 1.142 24-May-2007  lukem Revert previous "64bit fix" because it actually removed desired functionality.
Instead, use the appropriate format specifier & cast to display the time_t.
 1.141 22-May-2007  tnn Unbreak debug printf on 64-bit arches.
 1.140 22-May-2007  lukem * main: call tzset() to ensure TZ is setup for other <time.h> functions.
* remotemodtime(): use strptime() to parse the reply.
* fetch_url(): ensure struct tm is zeroed before calling strptime().
 1.139 15-May-2007  lukem * Modify parse_url() to consistently strip the leading `/' off ftp URLs.
Fixes PR 17617.
* Use 'RFCnnnn' (with leading 0) instead of 'RFC nnnn', to be
consistent with the style in the RFC index.
* Refer to RFC3916 instead of 1738 or 2732.
* Expand the list of supported RFCs in ftp(1) to contain the document
name as well.
 1.138 17-Apr-2007  lukem * Implement -s srcaddr; uses srcaddr as the local IP address for all
connections.
Based on code in the version of ftp that FreeBSD had before they
replaced it with lukemftp.
* Move error message handling into ftp_connect() rather than in the
caller, so that more specific error reporting can occur.
* Improve consistency of various warning and error messages.
 1.137 12-Apr-2007  lukem whitespace pedantry
 1.136 11-Apr-2007  lukem getpass() can return NULL upon error in some implementations
(as documented in older standards documents, before the API was obsoleted).
Problem observed in tnftp on Solaris by Emil Mikulic.
 1.135 23-May-2006  jnemeth Coverity CID 3610: derefencing NULL pointer.
 1.134 28-Apr-2006  christos branches: 1.134.2;
Coverity CID 873: Only align if we printed a string.
 1.133 31-Jan-2006  christos rename debug to ftp_debug. grr libssh.
 1.132 31-Jan-2006  christos Rename xfoo() to ftp_foo() to avoid collisions with libssh. Don't ask.
 1.131 29-Jun-2005  christos forgot to remove the file pointer from the macro.
 1.130 29-Jun-2005  christos Add NO_USAGE and NO_DEBUG so that we can fit in the floppies again.
 1.129 10-Jun-2005  lukem Implement:
int getline(FILE *stream, char *buf, size_t buflen, const char **errormsg)
Read a line from the FILE stream into buf/buflen using fgets(), so up
to buflen-1 chars will be read and the result will be NUL terminated.
If the line has a trailing newline it will be removed.
If the line is too long, excess characters will be read until
newline/EOF/error.
Various -ve return values indicate different errors, and errormsg
will be changed to an error description if it's not NULL.

Convert to use getline() instead of fgets() whenever reading user input
to ensure that an overly long input line doesn't leave excess characters
for the next input operation to accidentally use as input.

Zero out the password & account after we've finished with it.

Consistently use getpass(3) (i.e, character echo suppressed) when
reading the account data. For some reason, historically the "login"
code suppressed echo for Account: yet the "user" command did not!

Display the hostname in the "getaddrinfo failed" warning.

Appease some -Wcast-qual warnings. Fixing all of these requires
significant code refactoring. (mmm, legacy code).
 1.128 27-May-2005  lukem formatbuf(): fix %m and %M to use the hostname, not the username.
 1.127 26-May-2005  lukem Improve method used in fileindir() to determine if `file' is in or under `dir':
realpath(3) on non-NetBSD systems may fail if the target filename doesn't
exist, so instead use realpath(3) on the parent directory of `file'.
Per discussion with Todd Eigenschink.
 1.126 19-May-2005  lukem tab cleanup
 1.125 19-May-2005  lukem Use size_t instead of int where appropriate.
 1.124 19-May-2005  lukem Some const cleanups.
 1.123 14-May-2005  dsl Helps if the definition of xconnect() matches its declaration....
 1.122 13-May-2005  lukem * Correct the "optlen" argument passed to getsockopt(3) and setsockopt(3)
in various places. Fixes a problem noted by Allen Briggs.
* Improve warning printed when connect(2) for the data channel fails.
 1.121 11-May-2005  lukem Use socklen_t instead of int as the 5th argument to getsockopt().
Improve invocation of setsockopt() and associated failure messages.
 1.120 07-May-2005  lukem correct a comment
 1.119 11-Apr-2005  lukem gratuitous whitespace cleanup (before someone else jumps the gun...)
 1.118 11-Apr-2005  lukem Implement a timeout on the accept(2) in dataconn() and the
connect(2) in xconnect() by temporarily setting O_NONBLOCK
on the socket and using xpoll() to wait for the operation
to succeed.
The timeout used is the '-q quittime' argument (defaults to
60s for accept(2), and the system default for connect(2)).
Idea inspired by discussion with Chuck Cranor.
This may (indirectly) fix various problems with timeouts
in active mode through broken firewalls.

Implement xpoll() as a wrapper around poll(2), to make it
easier to replace on systems without a functional poll(2).
Unconditionally use xpoll() instead of conditionally using
select(2) or poll(2).
 1.117 03-Jan-2005  lukem branches: 1.117.2;
Forbid filenames returned from mget that aren't in (or below) the
current directory.
The previous behaviour (of trusting the remote server's response when
retrieving the list of files to mget with prompting disabled) has been
in ftp ~forever, and has been a "known issue" for a long time.
Recently an advisory was published by D.J. Bernstein on behalf of
Yosef Klein warning of the problems with the previous behaviour, so
to alleviate concern I've fixed this with a sledgehammer.

Remember the local cwd after any operation which may change it.
Use "remotecwd" instead of "remotepwd".
 1.116 20-Jul-2004  lukem If an ftp auto-fetch transfer is interrupted by SIGINT (usually ^C),
exit with 130 instead of 1 (or rarely, 0).
This allows an ftp auto-fetch in a shell loop to correctly terminate the loop.
Should fix PR [pkg/26351], and possibly others.
 1.115 10-Apr-2004  lukem If connect(2) in xconnect() fails with EINTR, call select(2) on the socket
until it's writable or it fails with something other than EINTR.
This matches the behaviour in SUSv3, and prevents the problem when
pressing ^T (SIGINFO, which is marked as restartable) during connection
setup would cause ftp to fail with EADDRINUSE or EALREADY when the
second connect(2) was attempted on the same socket.
Problem found and solution provided by Maxime Henrion <mux@freebsd.org>.
 1.114 07-Aug-2003  agc Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22365, verified by myself.
 1.113 31-Jul-2003  lukem * Work around broken ftp servers (notably ProFTPd) that can't even follow
RFC 2389, and skip any amount of whitespace before a FEATure response.
The RFC says 'single space' yet ProFTPd puts two.
* Improve formatting of features[] debug dump.
 1.112 15-Jun-2003  lukem Don't coredump when printing '%n' in the prompt if there's no username yet.
Fix from Maxim Konovalov <maxim at freebsd dot org>.
 1.111 28-Feb-2003  lukem crank dates
 1.110 21-Jan-2003  jhawk Abstract out progress bar support into progressbar.[ch], for inclusion
in external programs (conditionalized on -DSTANDALONE_PROGRESS).

The following moved from util.c to progressbar.c:
alarmtimer(), progressmeter(), psummary(), ptransfer(),
xsignal(), xsignal_restart()

The following moved from extern.h and ftp_var.h to progressbar.h:
STALLTIME, verbose, fromatty, progress, quit_time, ttywidth
 1.109 27-Aug-2002  christos Add a -q <quit-time> flag to abort a transfer if it has stalled for <quit-time>
seconds. Ok'd by luke.
 1.108 08-Jun-2002  yamt make an auto const variable static.
 1.107 05-Jun-2002  lukem - when showing the final progress bar, replace "00:00 ETA" with the
elapsed time. (suggested by simonb)
- actually display transfer stats after a URL fetch. (bug introduced a
*long* time ago)
- update copyright & version
 1.106 26-Dec-2001  lukem branches: 1.106.2;
update copyrights
 1.105 19-Feb-2001  cgd convert to use getprogname()
 1.104 19-Feb-2001  lukem - display a limited progress bar (containing bytes xferred and xfer rate)
when the file size is unknown
- disable progress bar during remglob()
 1.103 15-Nov-2000  lukem - implement "mreget"; as per "mget" but uses "reget" instead of "get"
- add -N netrc and $NETRC, as methods to select an alternative .netrc file
- cache local user name and home directory for further use
- in mget(), use docase() instead of a local version to do the case
conversion.
 1.102 08-Sep-2000  lukem always include <netdb.h>, not just when INET6 is defined.
resolves PR [bin/10970] by Richard Earnshaw <rearnsha@cambridge.arm.com>>
 1.101 27-Aug-2000  lukem - in progressmeter() perform the check for foregroundproc() a little earlier
- removed unused variable `items' in list_vertical()
 1.100 06-Aug-2000  lukem * implement parseport(), which takes a string and attempts to convert
it to a numeric port number
* use parseport() in parse_url() and hookup()
* don't try and lookup the port number using getaddrinfo(), as it's too hard
to separate a failed host name lookup from a failed service name lookup.
this was causing lossage on systems that don't have `http' in services(5)
(such as solaris), but only crept in when we started using getaddrinfo()
unconditionally.
 1.99 01-Aug-2000  lukem - rename NO_QUAD to NO_LONG_LONG, QUAD* -> LL* and add ULL* (unsigned)
equivalents. name change suggested by Klaus Klein <kjk@netbsd.org>
- change defined(BSD4_4) || HAVE_SIN_LEN tests into HAVE_SOCKADDR_SA_LEN,
and set the latter if BSD4_4 exists
 1.98 30-Jul-2000  lukem clean up NO_QUAD support: create helper #defines and use as appropriate:
#define NOQUAD ! NOQUAD
------- ------ - ------
QUADF "%ld" "%lld"
QUADFP(x) "%" x "ld" "%" x "lld"
QUADT long long long
STRTOL(x,y,z) strtol(x,y,z) strtoll(x,y,z)
 1.97 18-Jul-2000  lukem add support for FEAT and OPTS commands with `features' and `opts'.
(from RFC 2389).

add support for MLST & MLSD (machine parseble listings) with 'mlst', 'mlsd'
and 'pmlsd' (mlsd |$PAGER) commands. (from draft-ietf-ftpext-mlst-11)

rename remotesyst() to getremoteinfo(), and modify to parse the result from
FEAT (if supported), and take into account the support for the various
extensions such as MDTM, SIZE, REST (STREAM), MLSD, and FEAT/OPTS.
put each feature into one of the following categories:
- known to work (explicit FEAT)
- unknown but assume works until explicit failure, when it's
then tagged as `known not to work'.
- known not to work (FEAT succeeded but didn't return anything,
or was unknown and then explicit failure)
assign results into features[] matrix.

add support to getreply() so that an optional callback will be called
for each line received from the server except for the first and last.
this is used in FEAT (and MLST) parsing.

modify various commands to check if REST (STREAM), MDTM and SIZE are
explicitly or implicitly supported before using.

fix `syst' when verbose is off.

minor knf (indent goto labels by one space, etc).

simply various command usage handlers by assuming that argv != NULL except
for quit() and disconnect().
 1.96 15-Jun-2000  lukem * migrate the SYST parsing from setpeer() into a separate remotesyst().
call remotesyst() only when login has been successful
some servers don't let you run SYST until you've successfully logged in.
* in fetch_ftp(), always call setpeer() with autologin disabled, and use
the following ftp_login() to DTRT.
this prevents ftp from trying to login a second time if the first autologin
fails when connecting to a remote site anonymously using autofetch.
* reset unix_proxy and unix_server in cleanuppeer()
* missed a function conversion in the KNF sweep...
 1.95 01-May-2000  lukem branches: 1.95.2;
convert to ANSI KNF
 1.94 01-May-2000  lukem * Add support for `fget localfile', which reads a list of filenames to
retrieve from localfile. Based on work by Darren Reed.
* Crank version.
* Update copyright dates.
 1.93 24-Apr-2000  itojun allow IPv6 extended numeric address in host part.
(draft-ietf-ipngwg-scopedaddr-format-01.txt)
fixes PR 9616.
 1.92 13-Apr-2000  lukem s/strtoq/strtoll/ (the latter is standardised)
 1.91 31-Jan-2000  lukem define private type `sigfunc' as
typedef void (*sigfunc) __P((int));
and replace use of sig_t and void (*)(int).

certain other OSes define sig_t differently to that (they add extra arguments),
and it causes problems due to function mismatches, etc...
 1.90 26-Jan-2000  lukem roll back to using sscanf() instead of strptime() to parse `yyyymmddhhmmss'
strings, since the latter technically can't parse dates without non
alphanumerics between the elements (even though netbsd's strptime() copes).
 1.89 26-Jan-2000  lukem put spaces between the % specifiers in the strptime format string
 1.88 25-Jan-2000  lukem be harsher about workaround
 1.87 25-Jan-2000  lukem work around lame ftpd's that don't return a correct post-Y2K date in
the output of `MDTM'.

obviously the programmer of aforementioned lame ftpd's did something like
"19%02d", tm->tm_year
instead of
"%04d", tm->tm_year + TM_YEAR_BASE

fixes [bin/9289] by jbernard@mines.edu
 1.86 03-Dec-1999  itojun fix memory leak in fetch_url (no freeaddrinfo was there).
sync with recent KAME.
 1.85 28-Nov-1999  lukem * implement xsl_init() and xsl_add(); error checking forms of sl_{init,add}()
* fix bug where the second press of <TAB> on an empty word (i.e, list
all options) may have resulted in an strncmp() against NULL. (detected
by _DIAGASSERT())
 1.84 27-Nov-1999  lukem in cleanuppeer(), reset username to NULL after free()ing it.
fixes [bin/8870] by Wolfgang Rupprecht <wolfgang@wsrcc.com>
 1.83 26-Nov-1999  lukem * complete_remote(): use remglob("", ...) instead of remglob(".", ...),
for listings of the current working directory; some ftp servers don't
like `NLST .'.
[noted by Giles Lean <giles@nemeton.com.au>]
* recvrequest(): treat remote=="" as remote==NULL when calling command().
(to support the above change)
* support `[user@]' in `[user@]host' and `[user@]host[:][path]'.
[based on idea (and initial code) from David Maxwell <david@fundy.ca>]
* `idle' may be invoked without any args
* reformat some comments
* reformat usage string in program and man page
* call updateremotepwd() after successful login, not after successful connect
* always call setsockopt(, IPPROTO_IP, IP_TOS, ) (et al); using #if
defined(IPPROTO_IP) doesn't work on certain foreign systems where
enums instead of #defines are used...
[noted by Matthias Pfaller <leo@dachau.marco.de>]
 1.82 12-Nov-1999  lukem - implement `set rprompt'; right side version of `set prompt'. depends on
EL_RPROMPT support i added to editline(3).
- allow $FTPPROMPT and $FTPRPROMPT to override defaults for the relevant
prompts
- move `%' formatting code from prompt() to expandbuf().
- implement `%.' and `%c', similar to the same % codes in tcsh(1)
(functionality I added to tcsh nearly 6 years ago), except that `%.'
always does `...trailing' and `%c' always does `/<x>trailing'.
- unknown `%foo' codes get printed as `%foo'
 1.81 11-Nov-1999  lukem - implement updateremotepwd(); update the global variable `remotepwd'
to contain the remote working directory.
- add `set prompt', a user configurable prompt. (defaults to `ftp> ').
the following escape characters a la tcsh(1) are supported: %/, %m,
%M, and %n.
- add global var `username'; used by prompt code
- fix a couple of minor memory leaks
- bump version
 1.80 09-Nov-1999  lukem prevent minor memory leak (unnecessary strdup)
 1.79 24-Oct-1999  lukem new features:
- add `usage'; displays the usage of a command.
implemented by calling the c_handler() with argc = 0, argv = "funcname".
- add `passive auto'; does the same as $FTPMODE=auto.
- add `set [option value]'; display all options, or set an option to a value.
- add `unset option'; unset an option.
- add getoptionvalue() to retrieve an option's value, and replace a few
global variables with calls to this.
- implement cleanuppeer(), which resets various bits of state back to
`disconnected'. call in disconnect() and lostpeer().
- support completing on `options'.
- improve recovery after a SIGINT may have closed the connection.
XXX: there's still a couple to fix

other stuff:
- various consistency fixes in the man page.
- ensure that the command usage strings in the code and man page match reality.
- mput/mget: check that the connection still exists before each xfer.
- minor cosmetic changes in confirm().
- set code correctly in sizecmd() and modtime()
- don't need \n in err() strings.
- change lostpeer to take an argument (rather than casting (sig_t)lostpeer
in signal handlers)
- knf and whitespace police.
 1.78 13-Oct-1999  lukem * fix up confirm() (broke `a' and `p' in last commit)
* simplify main loop (don't need `top' variable any more)
* use a struct sockaddr_in6.sin6_addr for the result from inet_pton(),
rather than u_char buf[16]
* add a few more comments
 1.77 12-Oct-1999  lukem a few user interface and cosmetic tweaks:
* confirm(): move from util.c to cmds.c. display mnemonic string in its prompt.
add support for `q' (terminate current xfer), `?' (show help list)
* in various signal handlers, output a linefeed only if fromatty.
* if fgets(stdin) returned NULL (i.e, EOF), clearerr(stdin) because you don't
want future fgets to fail. this is not done for the fgets() in the main
command loop, since ftp will quit at that point.
* unless ftp is invoked with -a, don't retain the anonftp setting between
hosts (`ftp somehost:' sets anonftp, but you don't want that to `stick'
if you close that connection and open a new one).
 1.76 11-Oct-1999  lukem * in the SIGINFO/SIGQUIT handler, print a newline before displaying
the xfer stats
* crankrate() should have a K&R decl (for consistency)
 1.75 09-Oct-1999  lukem * use sigsetjmp()/siglongjump() instead of setjmp()/longjmp(); the latter
don't save the signal mask on some foreign systems.
* ensure signal handlers don't use stdio and do reset errno if they
don't exit with siglongjmp()
* use a common SIGINT handler for {send,recv}request()
 1.74 07-Oct-1999  simonb Minor nit - don't append a '/' to tmpdir if it already ends in a '/'.
 1.73 05-Oct-1999  lukem * factor out SIGINFO setting into a handler that is always active (but only
prints out info if bytes > 0). only set the handler if SIGINFO is defined
* hijack SIGQUIT to be the same as SIGINFO (foreign ports have this, and it's
annoying to have SIGQUIT dump core on netbsd when it prints info on other
systems)
* in {recv,send}request(), factor a lot of duplicated code out into a
`cleanup' section at the end
* rework shell() a bit
 1.72 05-Oct-1999  lukem add TNFi copyright to all files i've done more than a minor amount of work to...
 1.71 05-Oct-1999  lukem enhancments from Marc Horowitz <marc@mit.edu> to improve connection timeouts:
* implement xsignal_restart(), which only sets the SA_RESTART flag if
specifically requested
* xsignal() is now a wrapper to xsignal_restart(). INFO, USR1, USR2 and WINCH
are restartable, ALRM, INT, PIPE and QUIT are not.
* improve getreply()'s timeout code to take advantage of the above.

other changes:
* improve wording of how globbing works for `classic' URLs (host:path).
suggested by John Refling <johnr@imageworks.com> in relation to PRs
[bin/8519] and [bin/8520]
* always compile in the `edit' command even if NO_EDITCOMPLETE defined.
it's just a no-op in the latter case, which is more consistent to
the users.
* always compile in about: support (i.e, remove NO_ABOUT).
i'm entitled to some vanity in this program...
* clean up some whitespace
 1.70 01-Oct-1999  lukem * don't bother with "pathnames.h"; it contained just a single #define.
instead, include <paths.h> in the files that need it
* slightly reorg ftp_var.h
 1.69 30-Sep-1999  lukem #define BUFLEFT is always used...
 1.68 29-Sep-1999  lukem * consistentnly use memset(a,0,c); there were some cases of memset(a,'\0',c)
* remove explicit extern int h_errno; it's in <netdb.h>
* add <termios.h> back to util.c; it contains struct winsize on some systems
 1.67 28-Sep-1999  lukem revert snprintf() -> sprintf(). the original idea was that sprintf() would be
more portable, except that certain systems have char *sprintf() instead of
int sprintf(). (we'll provide snprintf() on those)
 1.66 28-Sep-1999  lukem don't fudge out the stalltime when calculating the ETA. this means that the
ETA might be over and will appear to drop rapidly, rather than being under
and appear not to change. the original code makes sense when you're testing
by suspending & resuming the client. however, the unfudged number is probably
better in reality, especially for slow spurty networks.

requested by ITOH Yasufumi <itohy@netbsd.org> in [bin/7977]
 1.65 28-Sep-1999  lukem * add new commands:
lpage page local files
pdir as dir, but through your $PAGER
pls as ls, but through your $PAGER
* implement docase() (a la dotrans() et al) and use appropriately, rototilling
some duplicated code
* globulize(): modify to return a pointer to the strdup()ed result in all cases,
and hack the code that calls it to take this into account
* replace strcpy() and strncpy() with strlcpy()
* put(), getit(): use some aptly named local vars instead of argv[...]
* delint
 1.64 27-Sep-1999  lukem * idle(): rename to idlecmd(). certain linux distributions have an incompatible
prototype for idle() in <unistd.h> (which i thought was against namespace
and sensibility guidelines, but...)
* consistently use xsignal() instead of signal(). we get known behaviour
in all cases (SA_RESTART), which is good for some borken foreign systems.
* remove signal.h from most files; it's unnecessary now
* fetch_url(): use `long chunksize' instead of ssize_t; it's more portable, and
we're setting chunksize with strtol() anyway
* xsignal(): only use SA_RESTART if it exists. SunOS 4.x doesn't have it
but has the inverse (SA_INTERRUPT). the original function i was inspired
from had this support (lib/signal.c, W. Richard Stevens' `UNP 2nd ed Vol 1').
* remove <termios.h> from util.c; it should be unnecessary now
 1.63 26-Sep-1999  lukem remove now-unused variable left over from last commit.
(repeat after me; `test compile before commit...')
 1.62 26-Sep-1999  lukem Use tcgetpgrp(fd) instead of ioctl(fd, TIOCGPGRP, &var), because
Kimmo Suominen <kim@netbsd.org> informs me that it's more portable.
 1.61 26-Sep-1999  lukem * replace ifdefs against __SVR4 and __linux__ with DIRENT_MISSING_D_NAMLEN;
it's more portable and more obvious
* remove the mkgmtime() && HAVE_TIMEGM stuff:
a) why should netbsd have to define HAVE_TIMEGM to compile cleanly?
b) foreign compiles of ftp should just be linked with working
timegm function

a more portable version of this ftp client will be released as a 3rdparty
product; no use polluting our code with half-baked attempts...
 1.60 24-Sep-1999  lukem * use %lld instead of %qd to print out (long long) vars.
(slightly more portable; e.g, solaris supports this)
* remove some fluff (lint)
 1.59 24-Sep-1999  lukem * finish replacing snprintf() with sprintf(), for portability reasons.
add lots of comments about how to size up the buffers, and add extra
checks to hopefully ensure that there won't be an overflow (unless
someone modifies the length of the sprintf()s).
* as part of the above, slightly rework the way the `*' bar is calculated.
also fixes a display bug when > 160 stars were needed to be printed.
the maximum progress bar width at this time is 256.
* remove some code that checks the port that was #if 0-ed out as part of the
ipv6 migration; it's not going to be used again.
 1.58 22-Sep-1999  lukem branches: 1.58.2;
* add support for `xferbuf', which sets both `sndbuf' and `rcvbuf'
* document the above three commands
* rototill the way the sndbuf and rcvbuf work. remove resetsockbufsize()
* use the appropriate socket buffer size as the size of the buffer that
the read()/write() loops use. speeds up things in some cases.
 1.57 22-Sep-1999  lukem replace snprintf() with strlcpy(), strlcat(), or direct assignment
where appropriate. (strlc*() are easier to port to foriegn systems).

XXX; there's still a few snprintf's in the progress meter stuff to convert
 1.56 21-Sep-1999  lukem cast 4th arg to [gs]etsockopt() to (void *), to satisfy some foriegn systems
 1.55 12-Jul-1999  lukem * change fetch_ftp() to be fully rfc 1738 compliant; if the URL contains
the an empty directory (e.g, between `some' and `path' in
`ftp://host/some//path'), then execute `CWD ' (without a path).
This command will probably fail on rfc 959 compliant servers, so
issue a warning in this case and bail. [noted by cgd].
(i wonder if the people who wrote rfc 1738 actually realised that this
requirement appears to contravene the spec for `cwd' in rfc 959 ?)
* replace isurl() with isipv6addr(), and use appropriately. fixes
auto-login with `classic ftp URLs' (e.g, `ftp somehost:')
* cleanup and rework some of the ipv6 stuff in parse_url()
* prevent potential coredump in fetch_ftp() when parsing `;type=X'
* KNF a few lines
* fix a couple of comments
* cleanup the man page a bit
 1.54 02-Jul-1999  itojun add dual-stack (IPv4/v6) support. hope I broke no other part...
 1.53 29-Jun-1999  lukem [fear this; more ftp hacking from lukem :-]

features:
 1.52 26-Jun-1999  lukem always compile in ptransfer() (and bits of progressmeter()), so that
transfer stats are printed when verbose is enabled even if ftp is compiled
with -DNO_PROGRESS/-DSMALL. this adds back some feedback that existed
in traditional ftp that was lost when i added the progress meter. (noticed
when ftp-ing from the -DSMALL version of ftp that's on the 1.4 boot floppy)
 1.51 24-Jun-1999  christos Avoid free'ing junk. It pisses off gnumalloc.
 1.50 20-Jun-1999  cgd split the "SMALL" #ifdefs into ones for NO_ABOUT, NO_EDITCOMPLETE, and
NO_PROGRESS. -DSMALL still implies all of those. progress meter support
isn't necessary for the smallest possible ftp client, but it adds very
little space and makes users' lives much better. Therefore, it should
be enabled for installation media if at all possible.
 1.49 02-Jun-1999  lukem * fix gate mode to login as `user@realhost' rather than using PASSERVE;
the latter only seemed to work for TIS Gauntlet and not TIS fwtk.
thanks to simonb@netbsd.org for testing this. fixes [bin/5556].
* if EOF (e.g, ^D) is entered at a username/password/account prompt which
happens to use fgets(), exit the login rather than treating EOF as CR.
* don't use the comma operator where separate statements are valid
* always use snprintf to copy stuff into malloced buffers, just in case
typos creep in and mean that the buffer ends up being overflowed
 1.48 20-May-1999  matthias ftp_login would only return success for proxy connections after the last
change. Fix this by setting rval at the right place.
 1.47 05-May-1999  lukem * fix ftp_login() so that -a always logs in anonymously, even if a
.netrc entry exists for the host with a valid user. noted by
Frederick Bruckman <fb@enteract.com> in [bin/7477]
* fix a minor memory leak associated with calling ruserpass() with
an empty user, password, or account.
 1.46 08-Mar-1999  lukem branches: 1.46.2;
whitespace & copyright fixes
 1.45 12-Feb-1999  lukem always run progressmeter() even if it's not the foregroundproc(), as some
calculations need to occur to prevent the calculated xfer rate from being
too fast if ftp(1) is backgrounded for a while during the xfer.
bug found by thor lancelot simon <tls@netbsd.org>.
 1.44 07-Feb-1999  lukem ftpvis(): escape LF too
 1.43 07-Feb-1999  lukem changes to fix [bin/6951] by Peter Simons <simons@cys.de>:
* implement ftpvis(), which \ quotes the following in the given string:
SP, TAB, \, CR, "
* use ftpvis() in complete_ambiguous(), to escape characters in a word
which would confuse makeargv()/slurpstring().
 1.42 31-Jan-1999  lukem revert previous write->fwrite conversion in progressmeter(); stdio is bad in signal handlers, nkay?
 1.41 24-Jan-1999  lukem * -v enables verbose & progress, -V disables both
* set setvbuf(ttyout, NULL, _IOLBF, 0) and remove a bunch of fflush(ttyout).
* use fwrite() instead of write() for progressmeter (don't intermix stdio
with non stdio ops)
 1.40 05-Jan-1999  lukem whitespace fix
 1.39 01-Jan-1999  lukem improve previous, so that display rounding is done to binary boundaries
(1024 not 1000) because the prefix determination is done that way.
tested on various boundary conditions.
 1.38 01-Jan-1999  lukem add two digits of precision to the transfer rate, and always show at
least KB/s. instead of printing `5414 B/s' now print `5.41 KB/s'.
 1.37 29-Dec-1998  lukem be consistent with return (foo) style
 1.36 18-Nov-1998  itohy Temporary fix of UTC -> time_t conversion:
1. changed to use timegm(3) on NetBSD and
2. supply alternative code for other Unix-like OSs
(NetBSD ftp shall be portable for some reasons :-).

This fix closes PR #6448.

NOTE: This should be fixed again if a portable UTC to time_t
conversion method is specified in some standards.
 1.35 06-Nov-1998  christos char -> unsigned char
 1.34 28-Sep-1998  lukem * add transfer rate to progress meter. (suggested by hubert feyrer)
* fix up progress meter display for files once > ~20.47MB has been
transferred. (noted by darren reed)
* use xsignal() for SIGWINCH, so window resizes don't abort xfers on
drainbamaged systems.
* don't assume that TIOCGWINSZ returns a non zero width.
 1.33 08-Aug-1998  lukem * implement xsignal(); same semantics as signal() but uses sigaction
with an explicit SA_RESTART. (needed for portability)
* use xsignal() for SIGALRM and SIGINFO handlers
 1.32 08-Aug-1998  lukem missed a NO_QUAD situation. be consistent in use of ifndef NO_QUAD
 1.31 03-Aug-1998  lukem features:
* support $no_proxy, which is a comma or space separated list of
host[:port] elements for which proxying is to be disabled.
(asked for by cgd in [bin/5027])
* if $FTPANONPASS is defined, use that as the anon ftp password
(instead of "`whoami`@")
* allow http URL's without a filename as long as an output file
is specified.

other stuff:
* implement parse_url(), which breaks up a URL into its bits, and use.
* simplify url_get() and auto_fetch() to use parse_url() and to not
modify the supplied URL or a copy of it.
* implement xmalloc() and xstrdup(); error-checked malloc()/strdup()
* add more consistency to messages, quoting strings in output as `%s'
 1.30 29-Jul-1998  lukem check EL_EDITMODE after el_source and disable editing if necessary
 1.29 26-Jul-1998  mycroft const poisoning.
 1.28 22-Jul-1998  lukem * send 'Connection: close' in HTTP/1.1 headers, preventing the
remove server from using a persistent connection, which speeds
up such requests.
* support http 301 and 302 redirects
* rewrite guts of url_get() to use fparseln() et al instead of
read(s,&p,1)... enables each in the header to be parsed
as necessary
* rename login to ftp_login, to remove conflict with util.h::login
* cleanup verbose messages during http proxy requests
 1.27 10-Jul-1998  thorpej Add "sndbuf" and "rcvbuf" commands for setting the socket buffer sizes,
which in turn can allow the use of larger TCP windows. This is a work in
progress; there is not yet support for specifying global defaults or
user prefrences on a host/network basis.
 1.26 06-Jul-1998  mrg - use an array MAXHOSTNAMELEN+1 size to hold hostnames
- ensure hostname from gethostname() is nul-terminated in all cases
- minor KNF
- use MAXHOSTNAMELEN over various other values/defines
- be safe will buffers that hold hostnames
 1.25 19-Jun-1998  kleink GLOB_QUOTE is dead; per POSIX, backslash quoting of special characters being
enabled is the default behaviour.
 1.24 04-Jun-1998  lukem some fixes & enhancements from openbsd's ftp, with extra fixes by me:
* default to passive with active fallback. $FTPMODE modifies this behaviour.
-A forces active connection.
* support '-o outfile' for auto-fetched files. outfile can be a file,
`-' (for stdout), or '|command' (to output each file through command).
* support '-r waittime', which retries the connection after waittime seconds
if it fails.
* fix 'page file' when restart is non-zero.
* try all ip-addresses of a host in a http fetch (as the normal ftp fetch
does).

XXX: a ``broken pipe'' error sometimes occurs with -o '|command';
i haven't tracked this down yet.
 1.23 20-May-1998  christos - add <signal.h> since we are using signals.
- cast arg to is*() to unsigned char.
- don't require quad_t to exist to compile.
- adapt to the new libedit version.
 1.22 04-Feb-1998  christos Don't hardcode 1900
 1.21 20-Jan-1998  lukem cosmetic mods to ptransfer() (which displays transfer statistics for
SIGINFO and at the end of the transfer):
* display time taken as [[x days?] hh:]mm:ss
* display ETA as [hh:]mm:ss
* print ' (stalled)' if the transfer is stalled
 1.20 18-Jan-1998  lukem define MAX_IN_PORT_T and use instead of USHRT_MAX. suggested by cgd@netbsd.org
 1.19 18-Jan-1998  lukem * ensure buffer for username is initialised, so ^D on username prompt
doesn't use garbage for the username. from "Soren S. Jorvang" <soren@t.dk>
in [bin/4559]
* use in_port_t for ports, and USHRT_MAX instead of 0xffff
(from millert@openbsd.org)
* use `NULL' instead of `(.... *)0' where appropriate.
 1.18 12-Dec-1997  gwr Kill the "progress bar" stuff when compiled with -DSMALL
because progressmeter() uses floats which are unwanted
when this is built under src/distrib/utils/x_ftp
(Maybe -DNO_FLOAT would be better?)
 1.17 01-Nov-1997  lukem * in recvrequest(), ignore restart_point unless "RETR"ieving. fixes problems
where a remote completion or `mget' would confuse the client a `restart'
had been issued beforehand. now, `restart' is remembered until an operation
that can actually use it is invoked.
* in sendrequest(), don't reset restart_point upon entry. fixes `restart'
for `put' operations.
* if `restart' is invoked with no arguments, print current setting instead
of displaying a usage
* consistently use printf("%qd", (long long)restart_point) when displaying
restart_point
* use strto[lq]() instead of atol() when parsing `mark' and `restart' values
* remove unnecessary strlen()s when result of previous snprintf() will do
* replace a few malloc()/strcpy()s with strdup()s
* use SECSPERHOUR instead of '3600'
 1.16 14-Oct-1997  christos branches: 1.16.2;
PR/4257: Jaromir Dolecek: Compensate for the libedit changes.
 1.15 11-Oct-1997  mycroft If the ETA is too large to fit on the display (i.e. 100 hours or more), don't
try to display it.
Also, eliminate many extra calls to strlen().
 1.14 21-Sep-1997  lukem More cleanups from Todd Miller <Todd.Miller@courtesan.com>:
* Use an int, not "union wait".
* Move the "parsed_url" label so that the next statement is not an "else" --
some ansi C compilers don't like it the old way (SGI's for example).
* Deal with the possibility of getlogin(2) failing.
* Don't error out if the remote server doesn't support the "MDTM" command.
 1.13 23-Aug-1997  lukem support $TMPDIR
 1.12 18-Aug-1997  lukem bugs fixed:
* don't interpret '-' or '|' when a local filename is determined from
the remote name (i.e, in mget, and in get with only one argument).
This is implemented using an extra argument to recvrequest().
Fixes a major security hole.
* clean up memory leak when using globulize()
* clean up a couple of comments
* fix wording in TNF copyright

features added:
* support for TIS fwtk gate-ftp servers:
* read defaults from $FTPSERVER && $FTPSERVERPORT
* start in gate-ftp mode if invoked as 'gate-ftp'
* toggle or set with 'gate [host [port]]'
 1.11 21-Jul-1997  lukem fix compile on alpha:
* cast all %qd printfs to (long long) [inspired by billc@warped.net]
* replace sscanf %qd with strtoq() et al
 1.10 20-Jul-1997  lukem * use RCSID() && COPYRIGHT() macros
* cleanup for WARNS=1 (including some ugly '(void)&var' bits wrapped in
#ifdef __GNUC__ to shut up gcc warnings WRT setjmp/longjmp)
* use strtol() instead of atoi(), and more extensively check result of
conversion
* use u_int16_t instead of short or int for TCP port addresses
 1.9 10-Jun-1997  lukem * Only print the progress bar when ftp(1) is the foreground progress.
Patch from enami tsugutomo <enami@ba2.so-net.or.jp> in [bin/3735].
* Fix punctuation.
 1.8 12-May-1997  lukem don't print -ve ETA if file size has grown. noticed by hpeyerl@netbsd.org
 1.7 14-Apr-1997  lukem More enhancements/bugfixes (when will it end?)
* differentiate between being connected, and being logged in
* cleanup some text messages
* support username & password ftp URLs (ftp://user:pass@host/) in non-proxy
situations; assume proxy supports it for proxy situations.
* cd to / before performing any autofetch transfers
* use strncasecmp in URL parsing. fix from <Todd.Miller@courtesan.com>
 1.6 05-Apr-1997  lukem * support $ftp_proxy for ftp:// transfers [bin/3245]
* add "more" & "less" as synonyms for "page"
* move editline setup code into controlediting(), and call appropriately.
only setup setup terminal if going into interactive mode. inspired
by Todd Miller <todd.miller@courtesan.com>
 1.5 13-Mar-1997  lukem Features:
* support remglobbing of auto_fetch arguments
* new flag - '-e'; disable editing
* "page file" == "get file |${PAGER-less}"

Bugfixes/cleanup:
* consistently use a trailing '.' on messages
* code cleanup, including buffer overrun fixes, use puts
and putchar in places, etc (inspired by OpenBSD mods)
* disable progress bar when local-file is a pipe or '-'
* skip \r in http headers
* fix remote ftpd slash bug more elegantly (so it works with ////)
* abort_remote(): check if cout==NULL before using it. should fix [bin/3273]
* fixed up cosmetic problems when complete_remote() generated errors from the
remote server (such as "no files found", "login with user and pass", ...)
done by adding extra argument to remglob(), which is a pointer to an error
buffer to put messages in rather than printing to stdout.
 1.4 01-Feb-1997  lukem don't display negative ETA upon SIGINFO because filesize isn't known
 1.3 01-Feb-1997  lukem [Yet Another Huge Ftp Commit - hopefully the last for a while,
barring any more little things people want added ...]

New features:
* progressmeter is now asynchronous, so "stalled" transfers can be
detected. "- stalled -" is displayed instead of the ETA in this case.
When the xfer resumes, the time that the xfer was stalled for is
factored out of the ETA. It is debatable whether this is better than
not factoring it out, but I like it this way (I.e, if it stalls for 8
seconds and the ETA was 30 seconds, when it resumes the ETA will still
be 30 seconds).
* verbosity can be disabled on the command line (-V), so that in auto-fetch
mode the only lines displayed will be a description of the file, and
the progress bar (if possible)
* if the screen is resized (and detected via the SIGWINCH signal), the
progress bar will rescale automatically.

Bugs fixed:
* progress bar will not use the last character on the line, as this can
cause problems on some terminals
* screen dimensions (via ioctl(TIOCWINSZ)) should use stdout not stdin
* progressmeter() used some vars before initialising them
* ^D will quit now. [fixes bin/3162]
* use hstrerror() to generate error message for host name lookup failure.
* use getcwd instead of getwd (it should have been OK, but why tempt fate?)
* auto-fetch transfers will always return a positive exit value upon failure
or interruption, relative to the file's position in argv[].
* remote completion of / will work, without putting a leading "///".
This is actually a bug in ftpd(1), where "NLST /" prefixes all names
with "//", but fixing every ftpd(1) is not an option...
 1.2 30-Jan-1997  thorpej Don't try to divide by zero in progressmeter().
From Thorsten Frueauf <frueauf@ira.uka.de>, PR #3160.
 1.1 19-Jan-1997  lukem New features:
* Command line editing via editline(3) library.
* Context sensitive command and file completion, including remote files.

Enhancements to auto-fetch feature:
* Support for http:// URLs using the http protocol, including proxy HTTP
support via $htty_proxy if it's defined.
* The connection is kept open between successive files on the same host.
(obviously, this does not count for http requests.)
* Return value of ftp is 0 on no error, or the offset in argv[] of the file
which failed (i.e., argv[x] failed, ftp returns x).
* If the path in an ftp URL or classic format line has a trailing '/',
cd to the path and enter interactive mode. Fixes [bin/3011], albiet
requiring the user to help ftp in determining the operation.

Other changes:
* '-P port' works for normal ftp, and is the default for all classic style
auto-fetch transfers and for ftp URLs that don't specify the port.
(previously it would just work for the first xfer.)
* Some code moved into separated files along logical divisions.
* Editing and completion can be compiled out with -DSMALLFTP.
 1.16.2.6 18-Jan-1999  cgd patch to fix 1.3.3's filename completion problems. Fixes PR#6647. (frueauf)
 1.16.2.5 23-Nov-1998  cgd pull up rev 1.36 from trunk (itohy)
 1.16.2.4 15-Nov-1998  cgd patch from Simon Burge to make this actually compile on the branch
 1.16.2.3 10-Nov-1998  cgd pull up rev(s) 1.19-1.35 from trunk and patch to work on 1.3 branch. (feyrer)
 1.16.2.2 14-Dec-1997  mellon Pull rev 1.18 up from trunk (gwr)
 1.16.2.1 18-Nov-1997  mellon Pull rev 1.17 up from trunk (lukem)
 1.46.2.3 12-Feb-2000  he Pull up revisions 1.87-1.90 (via patch, requested by he):
Work around lame ftp servers returning year 19100 for year 2000.
Fixes PR#9289.
 1.46.2.2 25-Jun-1999  cgd pull up rev 1.50 from trunk. (cgd)
 1.46.2.1 22-Jun-1999  perry pullup 1.46->1.49 (lukem)
 1.58.2.1 27-Dec-1999  wrstuden Pull up to last week's -current.
 1.95.2.1 23-Jun-2000  minoura Sync w/ netbsd-1-5-base.
 1.106.2.1 10-Feb-2003  jmc Pullup rev 1.110 (requested by grant in ticket #1115)
Separate progress reporting into a utility set of routines. Use
these to provide a standalone progress binary and integrate support
for this into sysinst.
 1.117.2.15 26-May-2006  ghen Pull up following revision(s) (requested by jnemeth in ticket #1339):
usr.bin/ftp/util.c: revision 1.135
Coverity CID 3610: derefencing NULL pointer.
 1.117.2.14 24-Jul-2005  tron branches: 1.117.2.14.2;
Pull up revision 1.131 (requested by lukem in ticket #606):
forgot to remove the file pointer from the macro.
 1.117.2.13 24-Jul-2005  tron Pull up revision 1.130 (requested by lukem in ticket #606):
Add NO_USAGE and NO_DEBUG so that we can fit in the floppies again.
 1.117.2.12 24-Jul-2005  tron Pull up revision 1.129 (requested by lukem in ticket #606):
Implement:
int getline(FILE *stream, char *buf, size_t buflen, const char **errormsg)
Read a line from the FILE stream into buf/buflen using fgets(), so up
to buflen-1 chars will be read and the result will be NUL terminated.
If the line has a trailing newline it will be removed.
If the line is too long, excess characters will be read until
newline/EOF/error.
Various -ve return values indicate different errors, and errormsg
will be changed to an error description if it's not NULL.
Convert to use getline() instead of fgets() whenever reading user input
to ensure that an overly long input line doesn't leave excess characters
for the next input operation to accidentally use as input.
Zero out the password & account after we've finished with it.
Consistently use getpass(3) (i.e, character echo suppressed) when
reading the account data. For some reason, historically the "login"
code suppressed echo for Account: yet the "user" command did not!
Display the hostname in the "getaddrinfo failed" warning.
Appease some -Wcast-qual warnings. Fixing all of these requires
significant code refactoring. (mmm, legacy code).
 1.117.2.11 24-Jul-2005  tron Pull up revision 1.128 (requested by lukem in ticket #606):
formatbuf(): fix %m and %M to use the hostname, not the username.
 1.117.2.10 24-Jul-2005  tron Pull up revision 1.127 (requested by lukem in ticket #606):
Improve method used in fileindir() to determine if `file' is in or under `dir':
realpath(3) on non-NetBSD systems may fail if the target filename doesn't
exist, so instead use realpath(3) on the parent directory of `file'.
Per discussion with Todd Eigenschink.
 1.117.2.9 24-Jul-2005  tron Pull up revision 1.126 (requested by lukem in ticket #606):
tab cleanup
 1.117.2.8 24-Jul-2005  tron Pull up revision 1.125 (requested by lukem in ticket #606):
Use size_t instead of int where appropriate.
 1.117.2.7 24-Jul-2005  tron Pull up revision 1.124 (requested by lukem in ticket #606):
Some const cleanups.
 1.117.2.6 24-Jul-2005  tron Pull up revision 1.123 (requested by lukem in ticket #606):
Helps if the definition of xconnect() matches its declaration....
 1.117.2.5 18-May-2005  snj Pull up revision 1.122 (requested by lukem in ticket #301):
* Correct the "optlen" argument passed to getsockopt(3) and setsockopt(3)
in various places. Fixes a problem noted by Allen Briggs.
* Improve warning printed when connect(2) for the data channel fails.
 1.117.2.4 18-May-2005  snj Pull up revision 1.121 (requested by lukem in ticket #318):
Use socklen_t instead of int as the 5th argument to getsockopt().
Improve invocation of setsockopt() and associated failure messages.
 1.117.2.3 09-May-2005  tron Pull up revision 1.120 (requested by lukem in ticket #266):
correct a comment
 1.117.2.2 09-May-2005  tron Pull up revision 1.119 (requested by lukem in ticket #266):
gratuitous whitespace cleanup (before someone else jumps the gun...)
 1.117.2.1 09-May-2005  tron Pull up revision 1.118 (requested by lukem in ticket #265):
Implement a timeout on the accept(2) in dataconn() and the
connect(2) in xconnect() by temporarily setting O_NONBLOCK
on the socket and using xpoll() to wait for the operation
to succeed.
The timeout used is the '-q quittime' argument (defaults to
60s for accept(2), and the system default for connect(2)).
Idea inspired by discussion with Chuck Cranor.
This may (indirectly) fix various problems with timeouts
in active mode through broken firewalls.
Implement xpoll() as a wrapper around poll(2), to make it
easier to replace on systems without a functional poll(2).
Unconditionally use xpoll() instead of conditionally using
select(2) or poll(2).
 1.117.2.14.2.1 26-May-2006  ghen Pull up following revision(s) (requested by jnemeth in ticket #1339):
usr.bin/ftp/util.c: revision 1.135
Coverity CID 3610: derefencing NULL pointer.
 1.134.2.1 19-Jun-2006  chap Sync with head.
 1.143.4.1 09-Jan-2008  matt sync with HEAD
 1.144.6.1 18-May-2008  yamt sync with head.
 1.146.2.2 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.146.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.148.6.1 13-May-2009  jym Sync with HEAD.

Third (and last) commit. See http://mail-index.netbsd.org/source-changes/2009/05/13/msg221222.html
 1.148.4.1 20-Nov-2010  riz Pull up following revision(s) (requested by lukem in ticket #1370):
usr.bin/ftp/extern.h: revision 1.78
usr.bin/ftp/fetch.c: revision 1.192
usr.bin/ftp/util.c: revision 1.153
Parse HTTP 'Date' entries in the `C' locale rather than the user's.
Fix from [bin/42917] (with minor changes), from KAMADA Ken'ichi.
 1.155.6.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.155.6.2 30-Oct-2012  yamt sync with head
 1.155.6.1 17-Apr-2012  yamt sync with head
 1.156.2.1 17-Dec-2013  bouyer Apply patch, requested by tron in ticket #997:
usr.bin/ftp/Makefile patch
usr.bin/ftp/cmds.c patch
usr.bin/ftp/cmdtab.c patch
usr.bin/ftp/extern.h patch
usr.bin/ftp/fetch.c patch
usr.bin/ftp/ftp.1 patch
usr.bin/ftp/ftp.c patch
usr.bin/ftp/ftp_var.h patch
usr.bin/ftp/main.c patch
usr.bin/ftp/progressbar.c patch
usr.bin/ftp/ssl.c patch
usr.bin/ftp/ssl.h patch
usr.bin/ftp/util.c patch
usr.bin/ftp/version.h patch

Add HTTPS support to ftp(1).
 1.157.2.1 25-Feb-2013  tls resync with head
 1.158.22.3 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1763:

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/cmds.c up to 1.141
usr.bin/ftp/complete.c up to 1.47
usr.bin/ftp/domacro.c up to 1.23
usr.bin/ftp/extern.h up to 1.82
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp.c up to 1.174
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/progressbar.c up to 1.24
usr.bin/ftp/progressbar.h up to 1.9
usr.bin/ftp/ssl.c up to 1.12
usr.bin/ftp/ssl.h up to 1.5
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

ftp(1): validate address from PASV and LPSV response.
ftp(1): use raw write(2) instead of fwrite(3) to avoid stream
corruption because of the progress bar interrupts.
Fixes for PR 56219 and PR 55857.
PR 57003: Support relative redirects.
 1.158.22.2 12-Sep-2022  martin Backout ticket #1763 for now - trust anchors are not solved.
 1.158.22.1 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1763:

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/cmds.c up to 1.141
usr.bin/ftp/complete.c up to 1.47
usr.bin/ftp/domacro.c up to 1.23
usr.bin/ftp/extern.h up to 1.82
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp.c up to 1.174
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/progressbar.c up to 1.24
usr.bin/ftp/progressbar.h up to 1.9
usr.bin/ftp/ssl.c up to 1.11
usr.bin/ftp/ssl.h up to 1.5
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

ftp(1): validate address from PASV and LPSV response.
ftp(1): use raw write(2) instead of fwrite(3) to avoid stream
corruption because of the progress bar interrupts.
Fixes for PR 56219 and PR 55857.
PR 57003: Support relative redirects.
 1.159.4.1 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.160.2.4 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1523

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/ssl.c up to 1.12
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

PR 57003: Support relative redirects.
 1.160.2.3 12-Sep-2022  martin Back out ticket #1523 for now - trust anchor validation is not solved.
 1.160.2.2 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1523

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/ssl.c up to 1.11
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

PR 57003: Support relative redirects.
 1.160.2.1 14-Jun-2021  martin Pull up following revision(s) (requested by lukem in ticket #1291):

usr.bin/ftp/ftp.c: revision 1.169
usr.bin/ftp/util.c: revision 1.161

ftp: exit if lostpeer invoked by a signal

lostpeer() calls too many async-unsafe functions (both directly
and indirectly) to close and cleanup the remote connections,
so just exit after the cleanup if invoked by a signal.

Reported in private mail by Qi Hou.

May also resolve a crash reported by Thomas Klausner.
 1.164.2.4 02-Dec-2024  martin Pull up following revision(s) (requested by lukem in ticket #1021):

usr.bin/ftp/ftp.c: revision 1.178
usr.bin/ftp/version.h: revision 1.99
usr.bin/ftp/ruserpass.c: revision 1.35
usr.bin/ftp/main.c: revision 1.134
usr.bin/ftp/main.c: revision 1.135
usr.bin/ftp/progressbar.c: revision 1.27
usr.bin/ftp/util.c: revision 1.165
usr.bin/ftp/cmds.c: revision 1.144
usr.bin/ftp/extern.h: revision 1.84
usr.bin/ftp/fetch.c: revision 1.242
usr.bin/ftp/ftp.1: revision 1.160

s/bninary/binary/ in comment.

extract duplicate code into a function.

Check bounds when copying to destination.

Remove const where the const string ended up being overwritten.

use unsigned when doing shifts.

remove const when string gets overwritten.

ftp: exit non-zero if short http transfer when filesize is known
If a http file size is known and the fetch finishes with less bytes
transferred, exit non-zero.
Bump version to 20241129.
PR bin/54713
PR bin/58281

ftp: help improvements
Document -? as a separate mode.
Document -H HEADER in the usage.
Clarify units for -b and -x.
Consistent argument names between ftp -? and ftp(1).

ftp: order getopt Upper before lower
Consistently order options in getopt and the switch
with the upper case option before the lower case option.
This makes it easier to cross-reference with -? and ftp(1).
No functional change.
 1.164.2.3 13-Oct-2024  martin Pull up following revision(s) (requested by riastradh in ticket #970):

tests/usr.bin/Makefile: revision 1.42
usr.bin/ftp/ruserpass.c: revision 1.34
usr.bin/ftp/main.c: revision 1.130
usr.bin/ftp/ssl.c: revision 1.17
usr.bin/ftp/main.c: revision 1.131
usr.bin/ftp/ssl.c: revision 1.18
usr.bin/ftp/main.c: revision 1.132
usr.bin/ftp/ssl.c: revision 1.19
usr.bin/ftp/main.c: revision 1.133
distrib/sets/lists/tests/mi: revision 1.1342
usr.bin/ftp/ftp.1: revision 1.151
usr.bin/ftp/ftp.1: revision 1.152
usr.bin/ftp/progressbar.c: revision 1.25
usr.bin/ftp/ftp.1: revision 1.153
usr.bin/ftp/progressbar.c: revision 1.26
usr.bin/ftp/ftp.1: revision 1.155
usr.bin/ftp/ftp.1: revision 1.156
usr.bin/ftp/fetch.c: revision 1.239
usr.bin/ftp/ftp.1: revision 1.157
usr.bin/ftp/ftp.1: revision 1.158
usr.bin/ftp/ftp.1: revision 1.159
usr.bin/ftp/ftp_var.h: revision 1.87
etc/mtree/NetBSD.dist.tests: revision 1.208
usr.bin/ftp/ftp_var.h: revision 1.88
usr.bin/ftp/ftp_var.h: revision 1.89
usr.bin/ftp/cmds.c: revision 1.142
usr.bin/ftp/util.c: revision 1.168
usr.bin/ftp/cmds.c: revision 1.143
tests/usr.bin/ftp/custom_headers.sh: revision 1.1
usr.bin/ftp/ssl.c: revision 1.20
usr.bin/ftp/complete.c: revision 1.48
tests/usr.bin/ftp/Makefile: revision 1.1
tests/usr.bin/ftp/t_custom_headers.sh: revision 1.1
usr.bin/ftp/fetch.c: revision 1.240
usr.bin/ftp/fetch.c: revision 1.241
usr.bin/ftp/ftp.c: revision 1.176
usr.bin/ftp/ftp.c: revision 1.177
(all via patch)

ftp(1): wording and formatting improvements

Fix grammar issue with "Support values" reported in private mail.
Document all file transfer types in "type" and cross-reference that.
Consistency fixes in describing file transfer parameters and types.

Fix some mandoc -Tlint issues (except "useless macro: Tn").

Add -b <buflen> to specify the buffer size.

ftp: bump FTPBUFLEN from 4kB to 16kB
sourceforge.net returns a 5kB content-security-policy.
Analyzed by mlelstv@ who reports usual limits are between 4kB and 48kB.
default is now 16K

ftp: improve -b documentation

Order -b bufsize in the synopsis.

Document the actual default value.

ftp: improve units used in comments and errors
Use "KiB" instead of "K" in errors.
Clarify related comments.

pass some lint.

PR/58581: Sunil Nimmagadda: Add flag to allow specifying extra http header
fields.

ftp(1): Nix trailing whitespace in man page.
No functional change intended.

PR bin/58581: ftp(1) should allow specifying header fields in http requests
fix markup (h -> H), explain about multiple headers, fix usage (from RVP)

Don't forget the dot, use the intended macro name (I think),
and improve the wording a little. (All related to the -H option.)
ftp(1): Add test for custom HTTP header fields.

Based on a patch from Sunil Nimmagadda.

PR bin/58581: ftp(1) should allow specifying header fields in http
requests
 1.164.2.2 16-May-2023  martin Pull up following revision(s) (requested by lukem in ticket #171):

usr.bin/ftp/ssl.c: revision 1.15
usr.bin/ftp/util.c: revision 1.167
usr.bin/ftp/ftp.c: revision 1.175
usr.bin/ftp/version.h: revision 1.97

add timeout for ssl connect

Implement a timeout for SSL connection setup, using -q QUITTIME,
defaulting to 60 seconds.

SSL_connect(3) (unlike connect(2)) doesn't timeout by default.
Adapt ssl error messages destination: if unexpected error
from local API, use warn()/warnx() to stderr;
if expected error from a network operation (e.g., timeouts),
use fprintf to ttyout (which might be stdout).

Consistently use ftp_poll() instead of select();
ssl.c (using select()) was added 7 years after the
previous uses of select() were converted to poll().

Check EAGAIN as well as existing EINTR error from ftp_poll(),
for portability.
 1.164.2.1 16-May-2023  martin Pull up following revision(s) (requested by lukem in ticket #170):

usr.bin/ftp/ssl.c: revision 1.13
usr.bin/ftp/fetch.c: revision 1.236
usr.bin/ftp/util.c: revision 1.166
usr.bin/ftp/main.c: revision 1.129
usr.bin/ftp/extern.h: revision 1.83
usr.bin/ftp/ftp.1: revision 1.148
usr.bin/ftp/cmdtab.c: revision 1.53
usr.bin/ftp/version.h: revision 1.96

Add option sslnoverify to control validation of SSL certificates.

Add netrc processing to fetch-mode (URL on command line) to enable options and autologin
via netrc.

Fix SSL cleanup in some error paths.

Certificate validation is now enabled by default. Set FTPSSLNOVERIFY=1 in environment
or configure a corresponding init macro via netrc to not validate certs (required if
you haven't installed a required CA certificate for OpenSSL).

Discussed with lukem@ on icb.
 1.167.2.1 02-Aug-2025  perseant Sync with HEAD
 1.1 10-Nov-1998  cgd branches: 1.1.2;
file util.h was initially added on branch netbsd-1-3.
 1.1.2.1 10-Nov-1998  cgd add new file (via patch) to back-port ftp to 1.3 branch. (feyrer)
 1.99 29-Nov-2024  lukem ftp: exit non-zero if short http transfer when filesize is known

If a http file size is known and the fetch finishes with less bytes
transferred, exit non-zero.

Bump version to 20241129.

PR bin/54713
PR bin/58281
 1.98 16-May-2023  lukem branches: 1.98.2;
ftp 20230516 for openssl 3.0 compat SSL_OP_IGNORE_UNEXPECTED_EOF
 1.97 05-May-2023  lukem add timeout for ssl connect

Implement a timeout for SSL connection setup, using -q QUITTIME,
defaulting to 60 seconds.
SSL_connect(3) (unlike connect(2)) doesn't timeout by default.

Adapt ssl error messages destination: if unexpected error
from local API, use warn()/warnx() to stderr;
if expected error from a network operation (e.g., timeouts),
use fprintf to ttyout (which might be stdout).

Consistently use ftp_poll() instead of select();
ssl.c (using select()) was added 7 years after the
previous uses of select() were converted to poll().

Check EAGAIN as well as existing EINTR error from ftp_poll(),
for portability.
 1.96 25-Feb-2023  mlelstv Add option sslnoverify to control validation of SSL certificates.
Add netrc processing to fetch-mode (URL on command line) to enable options and autologin
via netrc.
Fix SSL cleanup in some error paths.

Certificate validation is now enabled by default. Set FTPSSLNOVERIFY=1 in environment
or configure a corresponding init macro via netrc to not validate certs (required if
you haven't installed a required CA certificate for OpenSSL).

Discussed with lukem@ on icb.
 1.95 22-Sep-2022  lukem branches: 1.95.2;
update ftp version to 20220911

PR/57003: Handle relative URLs (patch by kim@)
 1.94 26-Aug-2021  lukem update ftp version to 20210826

Version bump for PASV/LPSV address validation
 1.93 03-Jun-2021  lukem use fetch_*() for I/O with SMALLPROG / !WITH_SSL builds

Adapt the SMALLPROG / -UWITH_SSL build to also use the fetch_*()
methods from ssl.c, instead of using stdio, as stdio isn't robust
when using interruptable signals.

Disable ssl-specific support in the fetch_*() methods if WITH_SSL
isn't defined, so SMALLPROG still doesn't have ssl support (as expected).

The resulting SMALLPROG binary is slightly larger than before
(e.g., 157KiB vs 153KiB on amd64).

Set version to 20210603 for this fix and the SO_KEEPALIVE fix for PR 56129.

PR install/56219
 1.92 06-Jan-2021  lukem branches: 1.92.4;
ftp: don't use restartable signals

Refactor to not rely upon restartable signals (SA_RESTART),
possibly fixing intermittent failures with -q QUITTIME.

ftp transfers: handle EINTR/EAGAIN in copy_bytes(),
instead of relying upon restartable signals.

http/https transfers: Explicitly print an error similar to
progressmeter() when timing-out for -Q QUITTIME in fetch_wait(),
and set errno to ETIMEDOUT so that the warn() in fetch_url()
prints a more accurate error message.

PR/55857
 1.91 18-Jul-2020  lukem ftp: add -? for help. improve synopsis

Add -? to display usage synopsis and help to stdout.
This allows for "ftp -? | less", which is more user friendly.
Errors still show usage to stderr.
Consistency improvements in some usage text.
 1.90 11-Jul-2020  lukem ftp.c: improve signal handler restoration

Only invoke the old signal handler if it's a real signal handler
and not SIG_IGN, SIG_DFL, SIG_HOLD, or SIG_ERR, using new static
function issighandler().
Avoids an intermittent race condition with a null pointer
dereference via (*SIG_DFL)().
Bug class reported by Joyu Liao from Juniper Networks.

Use SIG_ERR instead of NULL as the indicator that a signal handler
hasn't been changed, so that SIG_DFL (equivalent to NULL)
will be restored.
 1.89 08-Jun-2020  lukem ftp: exit if lostpeer invoked by a signal

lostpeer() calls too many async-unsafe functions (both directly
and indirectly) to close and cleanup the remote connections,
so just exit after the cleanup if invoked by a signal.

Reported in private mail by Qi Hou.
May also resolve a crash reported by Thomas Klausner.
 1.88 26-Feb-2020  lukem update ftp version to 20190622
 1.87 12-Sep-2015  wiz branches: 1.87.8; 1.87.16; 1.87.18;
Bump version for SNI support.
 1.86 23-Apr-2015  lukem Add -x xferbufsize to set xferbuf size.

Implement -x xferbufsize set the socket send and receive buffer size,
as per 'xferbuf' in interactive mode.

Patch from Nicholas Mills (via private mail), with minor adjustment by me.
 1.85 31-Oct-2014  lukem Version 20141026

Ignore special characters unless they're from the command line.
Fixes CVE-2014-8517
 1.84 05-May-2013  lukem branches: 1.84.6;
Version 20130220: restart fix, SYST response parsing improvement
 1.83 06-Feb-2013  christos merry belated x-mas.
 1.82 05-Jun-2010  lukem branches: 1.82.6; 1.82.8; 1.82.12; 1.82.14; 1.82.16;
In ftpvis(), prevent incomplete escape sequences at end of dst,
and ensure NUL-termination of dst. Also tweak for readibility.
Fix from Uwe Stuehler and Stefan Sperling, via Marc Balmer.
 1.81 05-Mar-2010  lukem ftp 20100305: fix http date parsing
 1.80 15-Nov-2009  lukem Crank version to 20090915:
* rename internal getline() to get_line() to avoid conflict with libc
* avoid NULL deref in verbose output in an error path
* improve man page markup
 1.79 24-Apr-2009  lukem Revert incorrect change made as part of the WARNS=4.
Fixes "ftp host [port]" on the CLI.
Noted by Geoff Wing.
 1.78 12-Apr-2009  lukem Fix numerous WARNS=4 issues (-Wcast-qual -Wsign-compare).
 1.77 13-Aug-2008  lukem branches: 1.77.2; 1.77.4; 1.77.6; 1.77.12;
Use AF_INET instead of AF_UNSPEC as the default family if !defined(INET6).
Avoids problem reported by Matthias Scheler <tron@NetBSD.org>.
 1.76 10-May-2008  skd Bump version string, as per lukem@.
 1.75 05-May-2008  lukem branches: 1.75.2;
Only attempt to el_parse() a command unknown by the default parser
if editing is enabled.
Fixes PR bin/38589
 1.74 28-Apr-2008  martin Remove clause 3 and 4 from TNF licenses
 1.73 22-Apr-2008  lukem Use the service name to getaddrinfo() (along with the host name), so that
features such as DNS Service Discovery have a better chance of working.
Suggested by David Young <dyoung>.

Display the service name in various status & error messages.

Don't getservbyname() the :port component of a URL; RFC 3986 says it's
just an unsigned number, not a service name.
 1.72 21-Jan-2008  lukem branches: 1.72.4;
Turn off the alarmtimer before resetting the SIGALRM handler back to SIG_DFL.
Fix from dieter roelants <dieter.NetBSD@pandora.be>
Should fix PR 35630.
 1.71 05-Dec-2007  lukem Rename HAVE_STRUCT_SOCKADDR_SA_LEN to HAVE_STRUCT_SOCKADDR_IN_SIN_LEN
to accurately reflect the structure member being used.
 1.70 22-Aug-2007  lukem branches: 1.70.2;
Improve parsing of chunked transfer chunks per RFC2616:
* more stringent chunk-size parsing
* ignore optional trailing ';chunk-ext' stuff, instead of barfing
* detect EOF before final \r\n.
 1.69 06-Aug-2007  lukem branches: 1.69.2;
Update version for:
Replace HAVE_SOCKADDR_SA_LEN with defined(HAVE_STRUCT_SOCKADDR_SA_LEN
 1.68 05-Jun-2007  lukem Enforce restriction that (http) proxied URL fetchs don't support
being restarted at this time.
PR #28697.
 1.67 24-May-2007  lukem Display times in RFC2822 form rather than using ctime(3), since
the former is more explicit about the timezone offset.
 1.66 22-May-2007  lukem * main: call tzset() to ensure TZ is setup for other <time.h> functions.
* remotemodtime(): use strptime() to parse the reply.
* fetch_url(): ensure struct tm is zeroed before calling strptime().
 1.65 15-May-2007  lukem * Modify parse_url() to consistently strip the leading `/' off ftp URLs.
Fixes PR 17617.
* Use 'RFCnnnn' (with leading 0) instead of 'RFC nnnn', to be
consistent with the style in the RFC index.
* Refer to RFC3916 instead of 1738 or 2732.
* Expand the list of supported RFCs in ftp(1) to contain the document
name as well.
 1.64 10-May-2007  lukem Implement copy_bytes() to copy bytes from one fd to another via the
provided buffer, with optional rate-limiting and hash-mark printing,
using one loop and handle short writes.
Refactor sendrequest() and recvrequest() to use copy_data().
Addresses PR 15943.
 1.63 18-Apr-2007  lukem Clean up use of confirm() and mbort() so that the current operation
is always passed in (instead of depending upon the 'mname' global).
For confirm(), if the second argument is NULL print the "Continue with <cmd>"
prompt. This fixes up the the display of interrupted prompts.
 1.62 17-Apr-2007  lukem * Implement -s srcaddr; uses srcaddr as the local IP address for all
connections.
Based on code in the version of ftp that FreeBSD had before they
replaced it with lukemftp.
* Move error message handling into ftp_connect() rather than in the
caller, so that more specific error reporting can occur.
* Improve consistency of various warning and error messages.
 1.61 12-Apr-2007  lukem Various fixes from Nicholas Marriott sent to openbsd-tech and in private email:
* Prevent segfault if a .netrc token is too long.
* Prevent segfault when using a macro from an empty macdef.
* Check more return values.
 1.60 11-Apr-2007  lukem Suppress printing non-COMPLETE reply strings from EPSV and EPRT, as we're
going to fall back to PASV / PORT (respectively) if the former fail,
and this avoids printing a failure reply followed by a success reply.
Should fix a problem with the emacs ftp wrapper.
 1.59 11-Apr-2007  lukem getpass() can return NULL upon error in some implementations
(as documented in older standards documents, before the API was obsoleted).
Problem observed in tnftp on Solaris by Emil Mikulic.
 1.58 26-Jul-2006  lukem If a file upload (via -u) fails, return an non-zero exit value based on the
index of the file that caused the problem (a la auto-fetch retrieval).
Problem noted by A P Garcia in private email.
 1.57 21-Aug-2005  lukem Don't clear the trailing character on the auth_url() username;
we now use getline() and that newline strips for us.
Problem found & fixed by Mark Davies.
 1.56 19-Jul-2005  lukem Revert back to using an int (instead of size_t) for holding a value that may
be negative.
Fixes progressbar display on terminals <43 columns wide.
Bug noted and solution suggested by Gavan Fantom.
 1.55 10-Jun-2005  lukem Implement:
int getline(FILE *stream, char *buf, size_t buflen, const char **errormsg)
Read a line from the FILE stream into buf/buflen using fgets(), so up
to buflen-1 chars will be read and the result will be NUL terminated.
If the line has a trailing newline it will be removed.
If the line is too long, excess characters will be read until
newline/EOF/error.
Various -ve return values indicate different errors, and errormsg
will be changed to an error description if it's not NULL.

Convert to use getline() instead of fgets() whenever reading user input
to ensure that an overly long input line doesn't leave excess characters
for the next input operation to accidentally use as input.

Zero out the password & account after we've finished with it.

Consistently use getpass(3) (i.e, character echo suppressed) when
reading the account data. For some reason, historically the "login"
code suppressed echo for Account: yet the "user" command did not!

Display the hostname in the "getaddrinfo failed" warning.

Appease some -Wcast-qual warnings. Fixing all of these requires
significant code refactoring. (mmm, legacy code).
 1.54 01-Jun-2005  lukem * Only print the "Trying <address>..." message if verbose and
there's more than one struct addrinfo in the getaddrinfo() result.
* Don't use non-standard "u_int".
 1.53 29-May-2005  lukem * fetch_ftp(): preserve 'anonftp' across a disconnect() so that multiple
ftp auto-fetches on the same command line login automatically.
* auto_fetch(): use an initialized volatile int to appease IRIX cc.
 1.52 27-May-2005  lukem formatbuf(): fix %m and %M to use the hostname, not the username.
 1.51 26-May-2005  lukem Improve method used in fileindir() to determine if `file' is in or under `dir':
realpath(3) on non-NetBSD systems may fail if the target filename doesn't
exist, so instead use realpath(3) on the parent directory of `file'.
Per discussion with Todd Eigenschink.
 1.50 14-May-2005  lukem Fix some cast issues highlighted by Scott Reynolds using gcc 4 on OSX.4
 1.49 13-May-2005  lukem * Correct the "optlen" argument passed to getsockopt(3) and setsockopt(3)
in various places. Fixes a problem noted by Allen Briggs.
* Improve warning printed when connect(2) for the data channel fails.
 1.48 11-May-2005  lukem Use socklen_t instead of int as the 5th argument to getsockopt().
Improve invocation of setsockopt() and associated failure messages.
 1.47 10-May-2005  lukem Prevent an overly-long input line causing a core dump when editing is enabled.
Issue noted by Ryoji Kanai in FreeBSD Problem Report # 77158.
 1.46 11-Apr-2005  lukem Implement a timeout on the accept(2) in dataconn() and the
connect(2) in xconnect() by temporarily setting O_NONBLOCK
on the socket and using xpoll() to wait for the operation
to succeed.
The timeout used is the '-q quittime' argument (defaults to
60s for accept(2), and the system default for connect(2)).
Idea inspired by discussion with Chuck Cranor.
This may (indirectly) fix various problems with timeouts
in active mode through broken firewalls.

Implement xpoll() as a wrapper around poll(2), to make it
easier to replace on systems without a functional poll(2).
Unconditionally use xpoll() instead of conditionally using
select(2) or poll(2).
 1.45 10-Apr-2005  lukem In fetch_url(), don't call freeaddrinfo(res0) too early, as we use pointers
to its contents later in the function.
Problem found by Onno van der Linden.
 1.44 12-Jan-2005  lukem branches: 1.44.2;
Fix ftp url reget when globs are being used.
Provided by Mathieu Arnold <mat@FreeBSD.org>.
 1.43 03-Jan-2005  lukem Forbid filenames returned from mget that aren't in (or below) the
current directory.
The previous behaviour (of trusting the remote server's response when
retrieving the list of files to mget with prompting disabled) has been
in ftp ~forever, and has been a "known issue" for a long time.
Recently an advisory was published by D.J. Bernstein on behalf of
Yosef Klein warning of the problems with the previous behaviour, so
to alleviate concern I've fixed this with a sledgehammer.

Remember the local cwd after any operation which may change it.
Use "remotecwd" instead of "remotepwd".
 1.42 10-Dec-2004  lukem * Always decode %xx in a url's user & pass components.
* Only remember {WWW,Proxy}-Authenticate "Basic" challenges; no point
in tracking any others since ftp doesn't support them.
* Improve the parsing of HTTP responses.
 1.41 08-Aug-2004  lukem Don't base64 encode the trailing NUL in the HTTP basic auth response.
Problem noted by Eric Haszlakiewicz.
 1.40 21-Jul-2004  lukem Slightly rework SIGINT handling; if we're exiting the auto-fetch stuff
and sigint_raised is non-zero, reset the handler for SIGINT to SIG_DFL
and raise(SIGINT) so that the appropriate wait(3) status is setup.
Based on solution proposed by Ognyan Kulev.
This should really fix PR [pkg/26351].
 1.39 20-Jul-2004  lukem Improve parsing of HTTP response headers to be more RFC2616 compliant, and
skip LWS (linear white space; CR, LF, space, tab) and the end of lines and
between the field name and the field value. This still isn't 100% compliant,
since we don't support "multi line" responses at this time.
This should fix PR [bin/22611] from TAMURA Kent (although I can't easily
find a http server to reproduce the problem against.)

Fix a minor memory leak when parsing HTTP response headers.
 1.38 20-Jul-2004  lukem Ensure that "mname" is set in ls() and mls() so that an aborted confirm()
prints the correct name.
Problem highlighted & suggested fix from PR [bin/17766] by Steve McClellan.
 1.37 20-Jul-2004  lukem If an ftp auto-fetch transfer is interrupted by SIGINT (usually ^C),
exit with 130 instead of 1 (or rarely, 0).
This allows an ftp auto-fetch in a shell loop to correctly terminate the loop.
Should fix PR [pkg/26351], and possibly others.
 1.36 15-Jul-2004  lukem Fix bug in Christos' recent cleanup which broke the "help" and "rhelp" commands.
 1.35 10-Apr-2004  lukem If connect(2) in xconnect() fails with EINTR, call select(2) on the socket
until it's writable or it fails with something other than EINTR.
This matches the behaviour in SUSv3, and prevents the problem when
pressing ^T (SIGINFO, which is marked as restartable) during connection
setup would cause ftp to fail with EADDRINUSE or EALREADY when the
second connect(2) was attempted on the same socket.
Problem found and solution provided by Maxime Henrion <mux@freebsd.org>.
 1.34 10-Dec-2003  lukem Don't warn about "ignored setsockopt" failures unless debugging is
enabled. Suggested by Todd Vierling.

Allow empty passwords in ftp://user:@host/file auto-fetch URLs,
per RFC 1738. Requested by Simon Poole.

Update version.
 1.33 31-Jul-2003  lukem crank version for:
* Work around broken ftp servers (notably ProFTPd) that can't even follow
RFC 2389, and skip any amount of whitespace before a FEATure response.
The RFC says 'single space' yet ProFTPd puts two.
Noted by DervishD <raul@pleyades.net>.
* Improve formatting of features[] debug dump.
* Invalidate remote directory completion cache if any command which
may change the remote contents completes successfully, including:
del, mdel, ren, mkdir, rmdir, quote, and all upload commands
Patch from Yar Tikhiy <yar@freebsd.org>.
 1.32 31-Jul-2003  lukem * $FTPUSERAGENT overrides the HTTP User-Agent header.
Based on patch from Douwe Kiela.
* Add about:tnftp
* Fix URL in about:netbsd
* Crank version
 1.31 15-Jun-2003  lukem Don't coredump when printing '%n' in the prompt if there's no username yet.
Fix from Maxim Konovalov <maxim at freebsd dot org>.
 1.30 28-Feb-2003  lukem crank version
 1.29 05-Jun-2002  lukem - when showing the final progress bar, replace "00:00 ETA" with the
elapsed time. (suggested by simonb)
- actually display transfer stats after a URL fetch. (bug introduced a
*long* time ago)
- update copyright & version
 1.28 06-May-2002  lukem - Only send port number in HTTP/1.1 Host: request if port != 80.
Fixes [bin/15415] from Takahiro Kambe <taca@sky.yamashina.kyoto.jp>
- Fix bad mode passed by mls() to recvrequest().
Fixes [bin/16642] from <steve.mcclellan@radisys.com>
 1.27 26-Dec-2001  lukem update copyrights
 1.26 23-Dec-2001  lukem Add -4 to force IPv4 and -6 to force IPv6 address usage.
From Hajimu UMEMOTO, via Mike Heffner of FreeBSD.

(FreeBSD has imported NetBSD's ftp as their ftp client;
Mike is sending back some of their local changes).
 1.25 20-Dec-2001  lukem Large file ASCII mode support by using fseeko() instead of fseek().
From Andrey A. Chernov of FreeBSD, via Mike Heffner.
 1.24 15-Dec-2000  lukem invoke cmdtab.c_handler()s with argv[0] == c_name instead of the
supplied name. that way the full (unambiguous) name is displayed in
error messages and usage strings.
 1.23 24-Nov-2000  itojun cope with 2553bis getnameinfo (always attach scope id)
getnameinfo error check.
 1.22 15-Nov-2000  lukem - implement "mreget"; as per "mget" but uses "reget" instead of "get"
- add -N netrc and $NETRC, as methods to select an alternative .netrc file
- cache local user name and home directory for further use
- in mget(), use docase() instead of a local version to do the case
conversion.
 1.21 28-Sep-2000  lukem explicitly use SOCK_STREAM with socket() instead of res->ai_socktype,
because it appears that linux with glibc doesn't set the latter
correctly after one of getaddrinfo() or getnameinfo().
 1.20 06-Aug-2000  lukem * implement parseport(), which takes a string and attempts to convert
it to a numeric port number
* use parseport() in parse_url() and hookup()
* don't try and lookup the port number using getaddrinfo(), as it's too hard
to separate a failed host name lookup from a failed service name lookup.
this was causing lossage on systems that don't have `http' in services(5)
(such as solaris), but only crept in when we started using getaddrinfo()
unconditionally.
 1.19 01-Aug-2000  lukem - rename NO_QUAD to NO_LONG_LONG, QUAD* -> LL* and add ULL* (unsigned)
equivalents. name change suggested by Klaus Klein <kjk@netbsd.org>
- change defined(BSD4_4) || HAVE_SIN_LEN tests into HAVE_SOCKADDR_SA_LEN,
and set the latter if BSD4_4 exists
 1.18 31-Jul-2000  lukem - we can't just rename BSD4_4 -> HAVE_SIN_LEN, since bsd systems define BSD4_4;
change tests to test for either defined(BSD4_4) or HAVE_SIN_LEN
- more KNF
 1.17 30-Jul-2000  lukem clean up NO_QUAD support: create helper #defines and use as appropriate:
#define NOQUAD ! NOQUAD
------- ------ - ------
QUADF "%ld" "%lld"
QUADFP(x) "%" x "ld" "%" x "lld"
QUADT long long long
STRTOL(x,y,z) strtol(x,y,z) strtoll(x,y,z)
 1.16 28-Jul-2000  lukem rename "opts" to "remopts", so people used to "o host" don't get bitten...
 1.15 18-Jul-2000  lukem add support for FEAT and OPTS commands with `features' and `opts'.
(from RFC 2389).

add support for MLST & MLSD (machine parseble listings) with 'mlst', 'mlsd'
and 'pmlsd' (mlsd |$PAGER) commands. (from draft-ietf-ftpext-mlst-11)

rename remotesyst() to getremoteinfo(), and modify to parse the result from
FEAT (if supported), and take into account the support for the various
extensions such as MDTM, SIZE, REST (STREAM), MLSD, and FEAT/OPTS.
put each feature into one of the following categories:
- known to work (explicit FEAT)
- unknown but assume works until explicit failure, when it's
then tagged as `known not to work'.
- known not to work (FEAT succeeded but didn't return anything,
or was unknown and then explicit failure)
assign results into features[] matrix.

add support to getreply() so that an optional callback will be called
for each line received from the server except for the first and last.
this is used in FEAT (and MLST) parsing.

modify various commands to check if REST (STREAM), MDTM and SIZE are
explicitly or implicitly supported before using.

fix `syst' when verbose is off.

minor knf (indent goto labels by one space, etc).

simply various command usage handlers by assuming that argv != NULL except
for quit() and disconnect().
 1.14 15-Jun-2000  lukem * migrate the SYST parsing from setpeer() into a separate remotesyst().
call remotesyst() only when login has been successful
some servers don't let you run SYST until you've successfully logged in.
* in fetch_ftp(), always call setpeer() with autologin disabled, and use
the following ftp_login() to DTRT.
this prevents ftp from trying to login a second time if the first autologin
fails when connecting to a remote site anonymously using autofetch.
* reset unix_proxy and unix_server in cleanuppeer()
* missed a function conversion in the KNF sweep...
 1.13 11-Jun-2000  lukem from itojun: better fix for previous (doesn't need in_addr_t or u_int32_t)
 1.12 11-Jun-2000  lukem portability fixes for lukemftp:
* initconn(): use in_addr_t instead of u_int32_t when manipulating IPv6
addresses (and assume anything with ipv6 has in_addr_t; if not, i'll
add an autoconf test for it)
* ai_unmapped(): not all systems have sin_len; so only set #ifdef BSD4_4
* fix some lint
 1.11 05-Jun-2000  lukem - fix ai_unmapped() to be a no-op in the !def INET6 case
- display `(-INET6)' at the end of the version string if !def INET6
- clarify in the man page that IPv6 support may not be present (for lukemftp :)
 1.10 31-May-2000  lukem Add support for 'ftp -u url file ...', to upload a list of files to given url.
Mostly based on [bin/10019] by Scott Aaron Bamford <sab@ansic.net>
 1.9 28-May-2000  lukem Change `ls' to use the `LIST' and not `NLST' FTP protocol command.
Now that after many years on not caring we find certain popular
ftp servers are starting to obey RFC959 to the letter of the law
and will only return a list of filenames (not directories or
other filetypes) in the output of `NLST', then `LIST' is more useful
in this case. (Note that the aforementioned pedanticness means that
filename completion isn't as useful as it could be...)
Fixes [bin/8937] by David A. Gatwood <dgatwood@deepspace.mklinux.org>
 1.8 01-May-2000  lukem branches: 1.8.2;
* Add support for `fget localfile', which reads a list of filenames to
retrieve from localfile. Based on work by Darren Reed.
* Crank version.
* Update copyright dates.
 1.7 13-Apr-2000  lukem crank version
 1.6 13-Apr-2000  lukem s/strtoq/strtoll/ (the latter is standardised)
 1.5 31-Jan-2000  lukem crank
 1.4 26-Jan-2000  lukem crank version
 1.3 25-Jan-2000  lukem crank version
 1.2 12-Dec-1999  lukem branches: 1.2.2;
crank version
 1.1 05-Dec-1999  lukem move version into separate file to reduce recompilation after version crank.
 1.2.2.1 27-Dec-1999  wrstuden Pull up to last week's -current.
 1.8.2.1 23-Jun-2000  minoura Sync w/ netbsd-1-5-base.
 1.44.2.13 28-Aug-2005  tron Pull up following revision(s) (requested by lukem in ticket #715):
usr.bin/ftp/version.h: revision 1.57
usr.bin/ftp/fetch.c: revision 1.164
Don't clear the trailing character on the auth_url() username;
we now use getline() and that newline strips for us.
Problem found & fixed by Mark Davies.
 1.44.2.12 24-Jul-2005  tron Pull up revision 1.56 (requested by lukem in ticket #606):
Revert back to using an int (instead of size_t) for holding a value that may
be negative.
Fixes progressbar display on terminals <43 columns wide.
Bug noted and solution suggested by Gavan Fantom.
 1.44.2.11 24-Jul-2005  tron Pull up revision 1.55 (requested by lukem in ticket #606):
Implement:
int getline(FILE *stream, char *buf, size_t buflen, const char **errormsg)
Read a line from the FILE stream into buf/buflen using fgets(), so up
to buflen-1 chars will be read and the result will be NUL terminated.
If the line has a trailing newline it will be removed.
If the line is too long, excess characters will be read until
newline/EOF/error.
Various -ve return values indicate different errors, and errormsg
will be changed to an error description if it's not NULL.
Convert to use getline() instead of fgets() whenever reading user input
to ensure that an overly long input line doesn't leave excess characters
for the next input operation to accidentally use as input.
Zero out the password & account after we've finished with it.
Consistently use getpass(3) (i.e, character echo suppressed) when
reading the account data. For some reason, historically the "login"
code suppressed echo for Account: yet the "user" command did not!
Display the hostname in the "getaddrinfo failed" warning.
Appease some -Wcast-qual warnings. Fixing all of these requires
significant code refactoring. (mmm, legacy code).
 1.44.2.10 24-Jul-2005  tron Pull up revision 1.54 (requested by lukem in ticket #606):
* Only print the "Trying <address>..." message if verbose and
there's more than one struct addrinfo in the getaddrinfo() result.
* Don't use non-standard "u_int".
 1.44.2.9 24-Jul-2005  tron Pull up revision 1.53 (requested by lukem in ticket #606):
* fetch_ftp(): preserve 'anonftp' across a disconnect() so that multiple
ftp auto-fetches on the same command line login automatically.
* auto_fetch(): use an initialized volatile int to appease IRIX cc.
 1.44.2.8 24-Jul-2005  tron Pull up revision 1.52 (requested by lukem in ticket #606):
formatbuf(): fix %m and %M to use the hostname, not the username.
 1.44.2.7 24-Jul-2005  tron Pull up revision 1.51 (requested by lukem in ticket #606):
Improve method used in fileindir() to determine if `file' is in or under `dir':
realpath(3) on non-NetBSD systems may fail if the target filename doesn't
exist, so instead use realpath(3) on the parent directory of `file'.
Per discussion with Todd Eigenschink.
 1.44.2.6 24-Jul-2005  tron Pull up revision 1.50 (requested by lukem in ticket #606):
Fix some cast issues highlighted by Scott Reynolds using gcc 4 on OSX.4
 1.44.2.5 18-May-2005  snj Pull up revision 1.49 (requested by lukem in ticket #301):
* Correct the "optlen" argument passed to getsockopt(3) and setsockopt(3)
in various places. Fixes a problem noted by Allen Briggs.
* Improve warning printed when connect(2) for the data channel fails.
 1.44.2.4 18-May-2005  snj Pull up revision 1.48 (requested by lukem in ticket #318):
Use socklen_t instead of int as the 5th argument to getsockopt().
Improve invocation of setsockopt() and associated failure messages.
 1.44.2.3 11-May-2005  tron Pull up revision 1.47 (requested by lukem in ticket #289):
Prevent an overly-long input line causing a core dump when editing is enabled.
Issue noted by Ryoji Kanai in FreeBSD Problem Report # 77158.
 1.44.2.2 09-May-2005  tron Pull up revision 1.46 (requested by lukem in ticket #265):
Implement a timeout on the accept(2) in dataconn() and the
connect(2) in xconnect() by temporarily setting O_NONBLOCK
on the socket and using xpoll() to wait for the operation
to succeed.
The timeout used is the '-q quittime' argument (defaults to
60s for accept(2), and the system default for connect(2)).
Idea inspired by discussion with Chuck Cranor.
This may (indirectly) fix various problems with timeouts
in active mode through broken firewalls.
Implement xpoll() as a wrapper around poll(2), to make it
easier to replace on systems without a functional poll(2).
Unconditionally use xpoll() instead of conditionally using
select(2) or poll(2).
 1.44.2.1 09-May-2005  tron Pull up revision 1.45 (requested by lukem in ticket #264):
In fetch_url(), don't call freeaddrinfo(res0) too early, as we use pointers
to its contents later in the function.
Problem found by Onno van der Linden.
 1.69.2.2 06-Aug-2007  lukem Update version for:
Replace HAVE_SOCKADDR_SA_LEN with defined(HAVE_STRUCT_SOCKADDR_SA_LEN
 1.69.2.1 06-Aug-2007  lukem file version.h was added on branch matt-mips64 on 2007-08-06 03:41:15 +0000
 1.70.2.2 23-Mar-2008  matt sync with HEAD
 1.70.2.1 09-Jan-2008  matt sync with HEAD
 1.72.4.1 18-May-2008  yamt sync with head.
 1.75.2.2 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.75.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.77.12.1 03-Nov-2014  msaitoh Pull up following revision(s) (requested by lukem in ticket #1929):
usr.bin/ftp/version.h: revision 1.85
Version 20141026
Ignore special characters unless they're from the command line.
Fixes CVE-2014-8517
 1.77.6.1 13-May-2009  jym Sync with HEAD.

Third (and last) commit. See http://mail-index.netbsd.org/source-changes/2009/05/13/msg221222.html
 1.77.4.1 03-Nov-2014  msaitoh Pull up following revision(s) (requested by lukem in ticket #1929):
usr.bin/ftp/version.h: revision 1.85
Version 20141026
Ignore special characters unless they're from the command line.
Fixes CVE-2014-8517
 1.77.2.1 03-Nov-2014  msaitoh Pull up following revision(s) (requested by lukem in ticket #1929):
usr.bin/ftp/version.h: revision 1.85
Version 20141026
Ignore special characters unless they're from the command line.
Fixes CVE-2014-8517
 1.82.16.1 03-Nov-2014  msaitoh Pull up following revision(s) (requested by lukem in ticket #1180):
usr.bin/ftp/version.h: revision 1.85
Version 20141026
Ignore special characters unless they're from the command line.
Fixes CVE-2014-8517
 1.82.14.1 03-Nov-2014  msaitoh Pull up following revision(s) (requested by lukem in ticket #1180):
usr.bin/ftp/version.h: revision 1.85
Version 20141026
Ignore special characters unless they're from the command line.
Fixes CVE-2014-8517
 1.82.12.2 23-Jun-2013  tls resync from head
 1.82.12.1 25-Feb-2013  tls resync with head
 1.82.8.3 27-Aug-2016  bouyer Apply patch, requested by nonaka in ticket #1375:
src/usr.bin/ftp/cmds.c: patch
src/usr.bin/ftp/fetch.c: patch
src/usr.bin/ftp/ftp.1: patch
src/usr.bin/ftp/ftp.c: patch
src/usr.bin/ftp/ftp_var.h: patch
src/usr.bin/ftp/main.c: patch
src/usr.bin/ftp/ssl.c: patch
src/usr.bin/ftp/ssl.h: patch
src/usr.bin/ftp/version.h: patch
Update ftp(1) to version 20150912, adding https via proxy support.
 1.82.8.2 03-Nov-2014  msaitoh Pull up following revision(s) (requested by lukem in ticket #1180):
usr.bin/ftp/version.h: revision 1.85
Version 20141026
Ignore special characters unless they're from the command line.
Fixes CVE-2014-8517
 1.82.8.1 17-Dec-2013  bouyer Apply patch, requested by tron in ticket #997:
usr.bin/ftp/Makefile patch
usr.bin/ftp/cmds.c patch
usr.bin/ftp/cmdtab.c patch
usr.bin/ftp/extern.h patch
usr.bin/ftp/fetch.c patch
usr.bin/ftp/ftp.1 patch
usr.bin/ftp/ftp.c patch
usr.bin/ftp/ftp_var.h patch
usr.bin/ftp/main.c patch
usr.bin/ftp/progressbar.c patch
usr.bin/ftp/ssl.c patch
usr.bin/ftp/ssl.h patch
usr.bin/ftp/util.c patch
usr.bin/ftp/version.h patch

Add HTTPS support to ftp(1).
 1.82.6.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.84.6.2 05-Nov-2015  riz Pull up following revision(s) (requested by wiz in ticket #981):
usr.bin/ftp/ftp.1: revision 1.135
usr.bin/ftp/ssl.c: revision 1.3
usr.bin/ftp/ssl.c: revision 1.4
usr.bin/ftp/ssl.h: revision 1.3
usr.bin/ftp/version.h: revision 1.86
usr.bin/ftp/version.h: revision 1.87
usr.bin/ftp/fetch.c: revision 1.207
usr.bin/ftp/main.c: revision 1.123
Add -x xferbufsize to set xferbuf size.
Implement -x xferbufsize set the socket send and receive buffer size,
as per 'xferbuf' in interactive mode.
Patch from Nicholas Mills (via private mail), with minor adjustment by me.
Add Server Name Indication (SNI) support for https.
Needed for e.g. some github URLs.
Bump version for SNI support.
servername cannot be NULL here.
Noted by joerg@.
 1.84.6.1 31-Oct-2014  martin Pull up following revision(s) (requested by lukem in ticket #172):
usr.bin/ftp/version.h: revision 1.85
Version 20141026
Ignore special characters unless they're from the command line.
Fixes CVE-2014-8517
 1.87.18.6 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1523

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/ssl.c up to 1.12
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

PR 57003: Support relative redirects.
 1.87.18.5 12-Sep-2022  martin Back out ticket #1523 for now - trust anchor validation is not solved.
 1.87.18.4 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1523

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/ssl.c up to 1.11
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

PR 57003: Support relative redirects.
 1.87.18.3 14-Jun-2021  martin Pull up following revision(s) (requested by lukem in ticket #1295):

usr.bin/ftp/ssl.c: revision 1.10
usr.bin/ftp/ssl.h: revision 1.5
usr.bin/ftp/version.h: revision 1.93
usr.bin/ftp/Makefile: revision 1.39

use fetch_*() for I/O with SMALLPROG / !WITH_SSL builds

Adapt the SMALLPROG / -UWITH_SSL build to also use the fetch_*()
methods from ssl.c, instead of using stdio, as stdio isn't robust
when using interruptable signals.

Disable ssl-specific support in the fetch_*() methods if WITH_SSL
isn't defined, so SMALLPROG still doesn't have ssl support (as expected).

The resulting SMALLPROG binary is slightly larger than before
(e.g., 157KiB vs 153KiB on amd64).

Set version to 20210603 for this fix and the SO_KEEPALIVE fix for PR 56129.

PR install/56219
 1.87.18.2 14-Jun-2021  martin Pull up following revision(s) (requested by lukem in ticket #1290):

usr.bin/ftp/version.h: revision 1.90
usr.bin/ftp/ftp.c: revision 1.170

ftp.c: improve signal handler restoration

Only invoke the old signal handler if it's a real signal handler
and not SIG_IGN, SIG_DFL, SIG_HOLD, or SIG_ERR, using new static
function issighandler().

Avoids an intermittent race condition with a null pointer
dereference via (*SIG_DFL)().

Bug class reported by Joyu Liao from Juniper Networks.

Use SIG_ERR instead of NULL as the indicator that a signal handler
hasn't been changed, so that SIG_DFL (equivalent to NULL)
will be restored.
 1.87.18.1 29-Jan-2021  martin Pull up following revision(s) (requested by lukem in ticket #1190):

usr.bin/ftp/progressbar.c: revision 1.24
usr.bin/ftp/ssl.c: revision 1.9
usr.bin/ftp/progressbar.h: revision 1.9
usr.bin/ftp/ftp.c: revision 1.171
usr.bin/ftp/version.h: revision 1.92

ftp: don't use restartable signals

Refactor to not rely upon restartable signals (SA_RESTART),
possibly fixing intermittent failures with -q QUITTIME.
ftp transfers: handle EINTR/EAGAIN in copy_bytes(),
instead of relying upon restartable signals.

http/https transfers: Explicitly print an error similar to
progressmeter() when timing-out for -Q QUITTIME in fetch_wait(),
and set errno to ETIMEDOUT so that the warn() in fetch_url()
prints a more accurate error message.

PR/55857
 1.87.16.1 08-Apr-2020  martin Merge changes from current as of 20200406
 1.87.8.3 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1763:

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/cmds.c up to 1.141
usr.bin/ftp/complete.c up to 1.47
usr.bin/ftp/domacro.c up to 1.23
usr.bin/ftp/extern.h up to 1.82
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp.c up to 1.174
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/progressbar.c up to 1.24
usr.bin/ftp/progressbar.h up to 1.9
usr.bin/ftp/ssl.c up to 1.12
usr.bin/ftp/ssl.h up to 1.5
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

ftp(1): validate address from PASV and LPSV response.
ftp(1): use raw write(2) instead of fwrite(3) to avoid stream
corruption because of the progress bar interrupts.
Fixes for PR 56219 and PR 55857.
PR 57003: Support relative redirects.
 1.87.8.2 12-Sep-2022  martin Backout ticket #1763 for now - trust anchors are not solved.
 1.87.8.1 12-Sep-2022  martin Catch up to current, requested by christos in ticket #1763:

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/cmds.c up to 1.141
usr.bin/ftp/complete.c up to 1.47
usr.bin/ftp/domacro.c up to 1.23
usr.bin/ftp/extern.h up to 1.82
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp.c up to 1.174
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/progressbar.c up to 1.24
usr.bin/ftp/progressbar.h up to 1.9
usr.bin/ftp/ssl.c up to 1.11
usr.bin/ftp/ssl.h up to 1.5
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

ftp(1): validate address from PASV and LPSV response.
ftp(1): use raw write(2) instead of fwrite(3) to avoid stream
corruption because of the progress bar interrupts.
Fixes for PR 56219 and PR 55857.
PR 57003: Support relative redirects.
 1.92.4.1 06-Jun-2021  cjep sync with head
 1.95.2.3 02-Dec-2024  martin Pull up following revision(s) (requested by lukem in ticket #1021):

usr.bin/ftp/ftp.c: revision 1.178
usr.bin/ftp/version.h: revision 1.99
usr.bin/ftp/ruserpass.c: revision 1.35
usr.bin/ftp/main.c: revision 1.134
usr.bin/ftp/main.c: revision 1.135
usr.bin/ftp/progressbar.c: revision 1.27
usr.bin/ftp/util.c: revision 1.165
usr.bin/ftp/cmds.c: revision 1.144
usr.bin/ftp/extern.h: revision 1.84
usr.bin/ftp/fetch.c: revision 1.242
usr.bin/ftp/ftp.1: revision 1.160

s/bninary/binary/ in comment.

extract duplicate code into a function.

Check bounds when copying to destination.

Remove const where the const string ended up being overwritten.

use unsigned when doing shifts.

remove const when string gets overwritten.

ftp: exit non-zero if short http transfer when filesize is known
If a http file size is known and the fetch finishes with less bytes
transferred, exit non-zero.
Bump version to 20241129.
PR bin/54713
PR bin/58281

ftp: help improvements
Document -? as a separate mode.
Document -H HEADER in the usage.
Clarify units for -b and -x.
Consistent argument names between ftp -? and ftp(1).

ftp: order getopt Upper before lower
Consistently order options in getopt and the switch
with the upper case option before the lower case option.
This makes it easier to cross-reference with -? and ftp(1).
No functional change.
 1.95.2.2 16-May-2023  martin Pull up following revision(s) (requested by lukem in ticket #171):

usr.bin/ftp/ssl.c: revision 1.15
usr.bin/ftp/util.c: revision 1.167
usr.bin/ftp/ftp.c: revision 1.175
usr.bin/ftp/version.h: revision 1.97

add timeout for ssl connect

Implement a timeout for SSL connection setup, using -q QUITTIME,
defaulting to 60 seconds.

SSL_connect(3) (unlike connect(2)) doesn't timeout by default.
Adapt ssl error messages destination: if unexpected error
from local API, use warn()/warnx() to stderr;
if expected error from a network operation (e.g., timeouts),
use fprintf to ttyout (which might be stdout).

Consistently use ftp_poll() instead of select();
ssl.c (using select()) was added 7 years after the
previous uses of select() were converted to poll().

Check EAGAIN as well as existing EINTR error from ftp_poll(),
for portability.
 1.95.2.1 16-May-2023  martin Pull up following revision(s) (requested by lukem in ticket #170):

usr.bin/ftp/ssl.c: revision 1.13
usr.bin/ftp/fetch.c: revision 1.236
usr.bin/ftp/util.c: revision 1.166
usr.bin/ftp/main.c: revision 1.129
usr.bin/ftp/extern.h: revision 1.83
usr.bin/ftp/ftp.1: revision 1.148
usr.bin/ftp/cmdtab.c: revision 1.53
usr.bin/ftp/version.h: revision 1.96

Add option sslnoverify to control validation of SSL certificates.

Add netrc processing to fetch-mode (URL on command line) to enable options and autologin
via netrc.

Fix SSL cleanup in some error paths.

Certificate validation is now enabled by default. Set FTPSSLNOVERIFY=1 in environment
or configure a corresponding init macro via netrc to not validate certs (required if
you haven't installed a required CA certificate for OpenSSL).

Discussed with lukem@ on icb.
 1.98.2.1 02-Aug-2025  perseant Sync with HEAD

RSS XML Feed