Home | History | Annotate | Download | only in ftp
History log of /src/usr.bin/ftp/util.c
RevisionDateAuthorComments
 1.168  25-Sep-2024  christos pass some lint.
 1.167  05-May-2023  lukem branches: 1.167.2;
add timeout for ssl connect

Implement a timeout for SSL connection setup, using -q QUITTIME,
defaulting to 60 seconds.
SSL_connect(3) (unlike connect(2)) doesn't timeout by default.

Adapt ssl error messages destination: if unexpected error
from local API, use warn()/warnx() to stderr;
if expected error from a network operation (e.g., timeouts),
use fprintf to ttyout (which might be stdout).

Consistently use ftp_poll() instead of select();
ssl.c (using select()) was added 7 years after the
previous uses of select() were converted to poll().

Check EAGAIN as well as existing EINTR error from ftp_poll(),
for portability.
 1.166  25-Feb-2023  mlelstv Add option sslnoverify to control validation of SSL certificates.
Add netrc processing to fetch-mode (URL on command line) to enable options and autologin
via netrc.
Fix SSL cleanup in some error paths.

Certificate validation is now enabled by default. Set FTPSSLNOVERIFY=1 in environment
or configure a corresponding init macro via netrc to not validate certs (required if
you haven't installed a required CA certificate for OpenSSL).

Discussed with lukem@ on icb.
 1.165  20-Jan-2023  andvar s/bninary/binary/ in comment.
 1.164  06-Aug-2022  andvar branches: 1.164.2;
s/blity/bility/ in various words, mainly in comments.
 1.163  27-Aug-2021  rillig ftp: fix undefined behavior when parsing FEAT

lint says: warning: argument to 'function from <ctype.h>' must be cast
to 'unsigned char', not to 'int' [342]
 1.162  25-Apr-2021  lukem better XXX comment for buggy ftp server
 1.161  08-Jun-2020  lukem ftp: exit if lostpeer invoked by a signal

lostpeer() calls too many async-unsafe functions (both directly
and indirectly) to close and cleanup the remote connections,
so just exit after the cleanup if invoked by a signal.

Reported in private mail by Qi Hou.
May also resolve a crash reported by Thomas Klausner.
 1.160  22-Jun-2019  christos branches: 1.160.2;
trim down error checking if we are small.
 1.159  20-Nov-2017  kre branches: 1.159.4;
Issue PWD commands to the server only when we actually
need the results, not speculatively, just in case we might.

Allows operation with some broken servers that get confused
by PWD commands in some situations, and saves server round
trips in the (modern) common case of
ftp ftp://path/name
where we never need to know the results from PWD.
 1.158  19-Feb-2013  dsl branches: 1.158.22;
When using the response to SYST to decide whether to default to 'binary'
be a lot less specific.
Kyocera printers report "230 Linux" but really don't want text transfers
of pdf files!
 1.157  04-Jul-2012  is branches: 1.157.2;
As discussed on tech-net@: Don't display expected EHOSTUNREACH for all but
the last connect attempts in terse mode.
 1.156  10-Dec-2011  lukem branches: 1.156.2;
Move determination of socket buffer sizes from startup to the first
time a socket is used, as the previous logic assumed AF_INET sockets
were available (which they may not be in an IPv6-only system).
Per discussion with Maxim Konovalov and the FreeBSD problem 162661.
 1.155  05-Jun-2010  lukem branches: 1.155.6;
In ftpvis(), prevent incomplete escape sequences at end of dst,
and ensure NUL-termination of dst. Also tweak for readibility.
Fix from Uwe Stuehler and Stefan Sperling, via Marc Balmer.
 1.154  05-Mar-2010  lukem Back to using 'RFC xxxx' instead of 'RFCxxxx'
 1.153  04-Mar-2010  lukem Parse HTTP 'Date' entries in the `C' locale rather than the user's.
Fix from [bin/42917] (with minor changes), from KAMADA Ken'ichi.
 1.152  13-Jul-2009  roy Rename internal getline() function to get_line() so it does
conflict with the soon to be added getline(3) libc function.
 1.151  15-Apr-2009  jld Another empty-body warning/error, another set of curly braces. This, and
the last one, appear to affect only the NO_DEBUG (e.g., SMALLPROG) build.
 1.150  12-Apr-2009  lukem Fix numerous WARNS=4 issues (-Wcast-qual -Wsign-compare).
 1.149  12-Apr-2009  lukem fix -Wshadow issues
 1.148  13-Aug-2008  lukem branches: 1.148.4; 1.148.6;
Use AF_INET instead of AF_UNSPEC as the default family if !defined(INET6).
Avoids problem reported by Matthias Scheler <tron@NetBSD.org>.
 1.147  10-May-2008  skd Add epsv6 and epsv to disable extended passive mode for ipv6 or both ipv4 and ipv6 respectively. This hack is due to our friends a Juniper Networks who break
epsv in ipv6. Should be fixed in ScreenOS 6.2.X.
 1.146  28-Apr-2008  martin branches: 1.146.2;
Remove clause 3 and 4 from TNF licenses
 1.145  22-Apr-2008  lukem Use the service name to getaddrinfo() (along with the host name), so that
features such as DNS Service Discovery have a better chance of working.
Suggested by David Young <dyoung>.

Display the service name in various status & error messages.

Don't getservbyname() the :port component of a URL; RFC 3986 says it's
just an unsigned number, not a service name.
 1.144  05-Dec-2007  lukem branches: 1.144.6;
DPRINTF() consistency tweaks
 1.143  24-May-2007  lukem branches: 1.143.4;
Display times in RFC2822 form rather than using ctime(3), since
the former is more explicit about the timezone offset.
 1.142  24-May-2007  lukem Revert previous "64bit fix" because it actually removed desired functionality.
Instead, use the appropriate format specifier & cast to display the time_t.
 1.141  22-May-2007  tnn Unbreak debug printf on 64-bit arches.
 1.140  22-May-2007  lukem * main: call tzset() to ensure TZ is setup for other <time.h> functions.
* remotemodtime(): use strptime() to parse the reply.
* fetch_url(): ensure struct tm is zeroed before calling strptime().
 1.139  15-May-2007  lukem * Modify parse_url() to consistently strip the leading `/' off ftp URLs.
Fixes PR 17617.
* Use 'RFCnnnn' (with leading 0) instead of 'RFC nnnn', to be
consistent with the style in the RFC index.
* Refer to RFC3916 instead of 1738 or 2732.
* Expand the list of supported RFCs in ftp(1) to contain the document
name as well.
 1.138  17-Apr-2007  lukem * Implement -s srcaddr; uses srcaddr as the local IP address for all
connections.
Based on code in the version of ftp that FreeBSD had before they
replaced it with lukemftp.
* Move error message handling into ftp_connect() rather than in the
caller, so that more specific error reporting can occur.
* Improve consistency of various warning and error messages.
 1.137  12-Apr-2007  lukem whitespace pedantry
 1.136  11-Apr-2007  lukem getpass() can return NULL upon error in some implementations
(as documented in older standards documents, before the API was obsoleted).
Problem observed in tnftp on Solaris by Emil Mikulic.
 1.135  23-May-2006  jnemeth Coverity CID 3610: derefencing NULL pointer.
 1.134  28-Apr-2006  christos branches: 1.134.2;
Coverity CID 873: Only align if we printed a string.
 1.133  31-Jan-2006  christos rename debug to ftp_debug. grr libssh.
 1.132  31-Jan-2006  christos Rename xfoo() to ftp_foo() to avoid collisions with libssh. Don't ask.
 1.131  29-Jun-2005  christos forgot to remove the file pointer from the macro.
 1.130  29-Jun-2005  christos Add NO_USAGE and NO_DEBUG so that we can fit in the floppies again.
 1.129  10-Jun-2005  lukem Implement:
int getline(FILE *stream, char *buf, size_t buflen, const char **errormsg)
Read a line from the FILE stream into buf/buflen using fgets(), so up
to buflen-1 chars will be read and the result will be NUL terminated.
If the line has a trailing newline it will be removed.
If the line is too long, excess characters will be read until
newline/EOF/error.
Various -ve return values indicate different errors, and errormsg
will be changed to an error description if it's not NULL.

Convert to use getline() instead of fgets() whenever reading user input
to ensure that an overly long input line doesn't leave excess characters
for the next input operation to accidentally use as input.

Zero out the password & account after we've finished with it.

Consistently use getpass(3) (i.e, character echo suppressed) when
reading the account data. For some reason, historically the "login"
code suppressed echo for Account: yet the "user" command did not!

Display the hostname in the "getaddrinfo failed" warning.

Appease some -Wcast-qual warnings. Fixing all of these requires
significant code refactoring. (mmm, legacy code).
 1.128  27-May-2005  lukem formatbuf(): fix %m and %M to use the hostname, not the username.
 1.127  26-May-2005  lukem Improve method used in fileindir() to determine if `file' is in or under `dir':
realpath(3) on non-NetBSD systems may fail if the target filename doesn't
exist, so instead use realpath(3) on the parent directory of `file'.
Per discussion with Todd Eigenschink.
 1.126  19-May-2005  lukem tab cleanup
 1.125  19-May-2005  lukem Use size_t instead of int where appropriate.
 1.124  19-May-2005  lukem Some const cleanups.
 1.123  14-May-2005  dsl Helps if the definition of xconnect() matches its declaration....
 1.122  13-May-2005  lukem * Correct the "optlen" argument passed to getsockopt(3) and setsockopt(3)
in various places. Fixes a problem noted by Allen Briggs.
* Improve warning printed when connect(2) for the data channel fails.
 1.121  11-May-2005  lukem Use socklen_t instead of int as the 5th argument to getsockopt().
Improve invocation of setsockopt() and associated failure messages.
 1.120  07-May-2005  lukem correct a comment
 1.119  11-Apr-2005  lukem gratuitous whitespace cleanup (before someone else jumps the gun...)
 1.118  11-Apr-2005  lukem Implement a timeout on the accept(2) in dataconn() and the
connect(2) in xconnect() by temporarily setting O_NONBLOCK
on the socket and using xpoll() to wait for the operation
to succeed.
The timeout used is the '-q quittime' argument (defaults to
60s for accept(2), and the system default for connect(2)).
Idea inspired by discussion with Chuck Cranor.
This may (indirectly) fix various problems with timeouts
in active mode through broken firewalls.

Implement xpoll() as a wrapper around poll(2), to make it
easier to replace on systems without a functional poll(2).
Unconditionally use xpoll() instead of conditionally using
select(2) or poll(2).
 1.117  03-Jan-2005  lukem branches: 1.117.2;
Forbid filenames returned from mget that aren't in (or below) the
current directory.
The previous behaviour (of trusting the remote server's response when
retrieving the list of files to mget with prompting disabled) has been
in ftp ~forever, and has been a "known issue" for a long time.
Recently an advisory was published by D.J. Bernstein on behalf of
Yosef Klein warning of the problems with the previous behaviour, so
to alleviate concern I've fixed this with a sledgehammer.

Remember the local cwd after any operation which may change it.
Use "remotecwd" instead of "remotepwd".
 1.116  20-Jul-2004  lukem If an ftp auto-fetch transfer is interrupted by SIGINT (usually ^C),
exit with 130 instead of 1 (or rarely, 0).
This allows an ftp auto-fetch in a shell loop to correctly terminate the loop.
Should fix PR [pkg/26351], and possibly others.
 1.115  10-Apr-2004  lukem If connect(2) in xconnect() fails with EINTR, call select(2) on the socket
until it's writable or it fails with something other than EINTR.
This matches the behaviour in SUSv3, and prevents the problem when
pressing ^T (SIGINFO, which is marked as restartable) during connection
setup would cause ftp to fail with EADDRINUSE or EALREADY when the
second connect(2) was attempted on the same socket.
Problem found and solution provided by Maxime Henrion <mux@freebsd.org>.
 1.114  07-Aug-2003  agc Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22365, verified by myself.
 1.113  31-Jul-2003  lukem * Work around broken ftp servers (notably ProFTPd) that can't even follow
RFC 2389, and skip any amount of whitespace before a FEATure response.
The RFC says 'single space' yet ProFTPd puts two.
* Improve formatting of features[] debug dump.
 1.112  15-Jun-2003  lukem Don't coredump when printing '%n' in the prompt if there's no username yet.
Fix from Maxim Konovalov <maxim at freebsd dot org>.
 1.111  28-Feb-2003  lukem crank dates
 1.110  21-Jan-2003  jhawk Abstract out progress bar support into progressbar.[ch], for inclusion
in external programs (conditionalized on -DSTANDALONE_PROGRESS).

The following moved from util.c to progressbar.c:
alarmtimer(), progressmeter(), psummary(), ptransfer(),
xsignal(), xsignal_restart()

The following moved from extern.h and ftp_var.h to progressbar.h:
STALLTIME, verbose, fromatty, progress, quit_time, ttywidth
 1.109  27-Aug-2002  christos Add a -q <quit-time> flag to abort a transfer if it has stalled for <quit-time>
seconds. Ok'd by luke.
 1.108  08-Jun-2002  yamt make an auto const variable static.
 1.107  05-Jun-2002  lukem - when showing the final progress bar, replace "00:00 ETA" with the
elapsed time. (suggested by simonb)
- actually display transfer stats after a URL fetch. (bug introduced a
*long* time ago)
- update copyright & version
 1.106  26-Dec-2001  lukem branches: 1.106.2;
update copyrights
 1.105  19-Feb-2001  cgd convert to use getprogname()
 1.104  19-Feb-2001  lukem - display a limited progress bar (containing bytes xferred and xfer rate)
when the file size is unknown
- disable progress bar during remglob()
 1.103  15-Nov-2000  lukem - implement "mreget"; as per "mget" but uses "reget" instead of "get"
- add -N netrc and $NETRC, as methods to select an alternative .netrc file
- cache local user name and home directory for further use
- in mget(), use docase() instead of a local version to do the case
conversion.
 1.102  08-Sep-2000  lukem always include <netdb.h>, not just when INET6 is defined.
resolves PR [bin/10970] by Richard Earnshaw <rearnsha@cambridge.arm.com>>
 1.101  27-Aug-2000  lukem - in progressmeter() perform the check for foregroundproc() a little earlier
- removed unused variable `items' in list_vertical()
 1.100  06-Aug-2000  lukem * implement parseport(), which takes a string and attempts to convert
it to a numeric port number
* use parseport() in parse_url() and hookup()
* don't try and lookup the port number using getaddrinfo(), as it's too hard
to separate a failed host name lookup from a failed service name lookup.
this was causing lossage on systems that don't have `http' in services(5)
(such as solaris), but only crept in when we started using getaddrinfo()
unconditionally.
 1.99  01-Aug-2000  lukem - rename NO_QUAD to NO_LONG_LONG, QUAD* -> LL* and add ULL* (unsigned)
equivalents. name change suggested by Klaus Klein <kjk@netbsd.org>
- change defined(BSD4_4) || HAVE_SIN_LEN tests into HAVE_SOCKADDR_SA_LEN,
and set the latter if BSD4_4 exists
 1.98  30-Jul-2000  lukem clean up NO_QUAD support: create helper #defines and use as appropriate:
#define NOQUAD ! NOQUAD
------- ------ - ------
QUADF "%ld" "%lld"
QUADFP(x) "%" x "ld" "%" x "lld"
QUADT long long long
STRTOL(x,y,z) strtol(x,y,z) strtoll(x,y,z)
 1.97  18-Jul-2000  lukem add support for FEAT and OPTS commands with `features' and `opts'.
(from RFC 2389).

add support for MLST & MLSD (machine parseble listings) with 'mlst', 'mlsd'
and 'pmlsd' (mlsd |$PAGER) commands. (from draft-ietf-ftpext-mlst-11)

rename remotesyst() to getremoteinfo(), and modify to parse the result from
FEAT (if supported), and take into account the support for the various
extensions such as MDTM, SIZE, REST (STREAM), MLSD, and FEAT/OPTS.
put each feature into one of the following categories:
- known to work (explicit FEAT)
- unknown but assume works until explicit failure, when it's
then tagged as `known not to work'.
- known not to work (FEAT succeeded but didn't return anything,
or was unknown and then explicit failure)
assign results into features[] matrix.

add support to getreply() so that an optional callback will be called
for each line received from the server except for the first and last.
this is used in FEAT (and MLST) parsing.

modify various commands to check if REST (STREAM), MDTM and SIZE are
explicitly or implicitly supported before using.

fix `syst' when verbose is off.

minor knf (indent goto labels by one space, etc).

simply various command usage handlers by assuming that argv != NULL except
for quit() and disconnect().
 1.96  15-Jun-2000  lukem * migrate the SYST parsing from setpeer() into a separate remotesyst().
call remotesyst() only when login has been successful
some servers don't let you run SYST until you've successfully logged in.
* in fetch_ftp(), always call setpeer() with autologin disabled, and use
the following ftp_login() to DTRT.
this prevents ftp from trying to login a second time if the first autologin
fails when connecting to a remote site anonymously using autofetch.
* reset unix_proxy and unix_server in cleanuppeer()
* missed a function conversion in the KNF sweep...
 1.95  01-May-2000  lukem branches: 1.95.2;
convert to ANSI KNF
 1.94  01-May-2000  lukem * Add support for `fget localfile', which reads a list of filenames to
retrieve from localfile. Based on work by Darren Reed.
* Crank version.
* Update copyright dates.
 1.93  24-Apr-2000  itojun allow IPv6 extended numeric address in host part.
(draft-ietf-ipngwg-scopedaddr-format-01.txt)
fixes PR 9616.
 1.92  13-Apr-2000  lukem s/strtoq/strtoll/ (the latter is standardised)
 1.91  31-Jan-2000  lukem define private type `sigfunc' as
typedef void (*sigfunc) __P((int));
and replace use of sig_t and void (*)(int).

certain other OSes define sig_t differently to that (they add extra arguments),
and it causes problems due to function mismatches, etc...
 1.90  26-Jan-2000  lukem roll back to using sscanf() instead of strptime() to parse `yyyymmddhhmmss'
strings, since the latter technically can't parse dates without non
alphanumerics between the elements (even though netbsd's strptime() copes).
 1.89  26-Jan-2000  lukem put spaces between the % specifiers in the strptime format string
 1.88  25-Jan-2000  lukem be harsher about workaround
 1.87  25-Jan-2000  lukem work around lame ftpd's that don't return a correct post-Y2K date in
the output of `MDTM'.

obviously the programmer of aforementioned lame ftpd's did something like
"19%02d", tm->tm_year
instead of
"%04d", tm->tm_year + TM_YEAR_BASE

fixes [bin/9289] by jbernard@mines.edu
 1.86  03-Dec-1999  itojun fix memory leak in fetch_url (no freeaddrinfo was there).
sync with recent KAME.
 1.85  28-Nov-1999  lukem * implement xsl_init() and xsl_add(); error checking forms of sl_{init,add}()
* fix bug where the second press of <TAB> on an empty word (i.e, list
all options) may have resulted in an strncmp() against NULL. (detected
by _DIAGASSERT())
 1.84  27-Nov-1999  lukem in cleanuppeer(), reset username to NULL after free()ing it.
fixes [bin/8870] by Wolfgang Rupprecht <wolfgang@wsrcc.com>
 1.83  26-Nov-1999  lukem * complete_remote(): use remglob("", ...) instead of remglob(".", ...),
for listings of the current working directory; some ftp servers don't
like `NLST .'.
[noted by Giles Lean <giles@nemeton.com.au>]
* recvrequest(): treat remote=="" as remote==NULL when calling command().
(to support the above change)
* support `[user@]' in `[user@]host' and `[user@]host[:][path]'.
[based on idea (and initial code) from David Maxwell <david@fundy.ca>]
* `idle' may be invoked without any args
* reformat some comments
* reformat usage string in program and man page
* call updateremotepwd() after successful login, not after successful connect
* always call setsockopt(, IPPROTO_IP, IP_TOS, ) (et al); using #if
defined(IPPROTO_IP) doesn't work on certain foreign systems where
enums instead of #defines are used...
[noted by Matthias Pfaller <leo@dachau.marco.de>]
 1.82  12-Nov-1999  lukem - implement `set rprompt'; right side version of `set prompt'. depends on
EL_RPROMPT support i added to editline(3).
- allow $FTPPROMPT and $FTPRPROMPT to override defaults for the relevant
prompts
- move `%' formatting code from prompt() to expandbuf().
- implement `%.' and `%c', similar to the same % codes in tcsh(1)
(functionality I added to tcsh nearly 6 years ago), except that `%.'
always does `...trailing' and `%c' always does `/<x>trailing'.
- unknown `%foo' codes get printed as `%foo'
 1.81  11-Nov-1999  lukem - implement updateremotepwd(); update the global variable `remotepwd'
to contain the remote working directory.
- add `set prompt', a user configurable prompt. (defaults to `ftp> ').
the following escape characters a la tcsh(1) are supported: %/, %m,
%M, and %n.
- add global var `username'; used by prompt code
- fix a couple of minor memory leaks
- bump version
 1.80  09-Nov-1999  lukem prevent minor memory leak (unnecessary strdup)
 1.79  24-Oct-1999  lukem new features:
- add `usage'; displays the usage of a command.
implemented by calling the c_handler() with argc = 0, argv = "funcname".
- add `passive auto'; does the same as $FTPMODE=auto.
- add `set [option value]'; display all options, or set an option to a value.
- add `unset option'; unset an option.
- add getoptionvalue() to retrieve an option's value, and replace a few
global variables with calls to this.
- implement cleanuppeer(), which resets various bits of state back to
`disconnected'. call in disconnect() and lostpeer().
- support completing on `options'.
- improve recovery after a SIGINT may have closed the connection.
XXX: there's still a couple to fix

other stuff:
- various consistency fixes in the man page.
- ensure that the command usage strings in the code and man page match reality.
- mput/mget: check that the connection still exists before each xfer.
- minor cosmetic changes in confirm().
- set code correctly in sizecmd() and modtime()
- don't need \n in err() strings.
- change lostpeer to take an argument (rather than casting (sig_t)lostpeer
in signal handlers)
- knf and whitespace police.
 1.78  13-Oct-1999  lukem * fix up confirm() (broke `a' and `p' in last commit)
* simplify main loop (don't need `top' variable any more)
* use a struct sockaddr_in6.sin6_addr for the result from inet_pton(),
rather than u_char buf[16]
* add a few more comments
 1.77  12-Oct-1999  lukem a few user interface and cosmetic tweaks:
* confirm(): move from util.c to cmds.c. display mnemonic string in its prompt.
add support for `q' (terminate current xfer), `?' (show help list)
* in various signal handlers, output a linefeed only if fromatty.
* if fgets(stdin) returned NULL (i.e, EOF), clearerr(stdin) because you don't
want future fgets to fail. this is not done for the fgets() in the main
command loop, since ftp will quit at that point.
* unless ftp is invoked with -a, don't retain the anonftp setting between
hosts (`ftp somehost:' sets anonftp, but you don't want that to `stick'
if you close that connection and open a new one).
 1.76  11-Oct-1999  lukem * in the SIGINFO/SIGQUIT handler, print a newline before displaying
the xfer stats
* crankrate() should have a K&R decl (for consistency)
 1.75  09-Oct-1999  lukem * use sigsetjmp()/siglongjump() instead of setjmp()/longjmp(); the latter
don't save the signal mask on some foreign systems.
* ensure signal handlers don't use stdio and do reset errno if they
don't exit with siglongjmp()
* use a common SIGINT handler for {send,recv}request()
 1.74  07-Oct-1999  simonb Minor nit - don't append a '/' to tmpdir if it already ends in a '/'.
 1.73  05-Oct-1999  lukem * factor out SIGINFO setting into a handler that is always active (but only
prints out info if bytes > 0). only set the handler if SIGINFO is defined
* hijack SIGQUIT to be the same as SIGINFO (foreign ports have this, and it's
annoying to have SIGQUIT dump core on netbsd when it prints info on other
systems)
* in {recv,send}request(), factor a lot of duplicated code out into a
`cleanup' section at the end
* rework shell() a bit
 1.72  05-Oct-1999  lukem add TNFi copyright to all files i've done more than a minor amount of work to...
 1.71  05-Oct-1999  lukem enhancments from Marc Horowitz <marc@mit.edu> to improve connection timeouts:
* implement xsignal_restart(), which only sets the SA_RESTART flag if
specifically requested
* xsignal() is now a wrapper to xsignal_restart(). INFO, USR1, USR2 and WINCH
are restartable, ALRM, INT, PIPE and QUIT are not.
* improve getreply()'s timeout code to take advantage of the above.

other changes:
* improve wording of how globbing works for `classic' URLs (host:path).
suggested by John Refling <johnr@imageworks.com> in relation to PRs
[bin/8519] and [bin/8520]
* always compile in the `edit' command even if NO_EDITCOMPLETE defined.
it's just a no-op in the latter case, which is more consistent to
the users.
* always compile in about: support (i.e, remove NO_ABOUT).
i'm entitled to some vanity in this program...
* clean up some whitespace
 1.70  01-Oct-1999  lukem * don't bother with "pathnames.h"; it contained just a single #define.
instead, include <paths.h> in the files that need it
* slightly reorg ftp_var.h
 1.69  30-Sep-1999  lukem #define BUFLEFT is always used...
 1.68  29-Sep-1999  lukem * consistentnly use memset(a,0,c); there were some cases of memset(a,'\0',c)
* remove explicit extern int h_errno; it's in <netdb.h>
* add <termios.h> back to util.c; it contains struct winsize on some systems
 1.67  28-Sep-1999  lukem revert snprintf() -> sprintf(). the original idea was that sprintf() would be
more portable, except that certain systems have char *sprintf() instead of
int sprintf(). (we'll provide snprintf() on those)
 1.66  28-Sep-1999  lukem don't fudge out the stalltime when calculating the ETA. this means that the
ETA might be over and will appear to drop rapidly, rather than being under
and appear not to change. the original code makes sense when you're testing
by suspending & resuming the client. however, the unfudged number is probably
better in reality, especially for slow spurty networks.

requested by ITOH Yasufumi <itohy@netbsd.org> in [bin/7977]
 1.65  28-Sep-1999  lukem * add new commands:
lpage page local files
pdir as dir, but through your $PAGER
pls as ls, but through your $PAGER
* implement docase() (a la dotrans() et al) and use appropriately, rototilling
some duplicated code
* globulize(): modify to return a pointer to the strdup()ed result in all cases,
and hack the code that calls it to take this into account
* replace strcpy() and strncpy() with strlcpy()
* put(), getit(): use some aptly named local vars instead of argv[...]
* delint
 1.64  27-Sep-1999  lukem * idle(): rename to idlecmd(). certain linux distributions have an incompatible
prototype for idle() in <unistd.h> (which i thought was against namespace
and sensibility guidelines, but...)
* consistently use xsignal() instead of signal(). we get known behaviour
in all cases (SA_RESTART), which is good for some borken foreign systems.
* remove signal.h from most files; it's unnecessary now
* fetch_url(): use `long chunksize' instead of ssize_t; it's more portable, and
we're setting chunksize with strtol() anyway
* xsignal(): only use SA_RESTART if it exists. SunOS 4.x doesn't have it
but has the inverse (SA_INTERRUPT). the original function i was inspired
from had this support (lib/signal.c, W. Richard Stevens' `UNP 2nd ed Vol 1').
* remove <termios.h> from util.c; it should be unnecessary now
 1.63  26-Sep-1999  lukem remove now-unused variable left over from last commit.
(repeat after me; `test compile before commit...')
 1.62  26-Sep-1999  lukem Use tcgetpgrp(fd) instead of ioctl(fd, TIOCGPGRP, &var), because
Kimmo Suominen <kim@netbsd.org> informs me that it's more portable.
 1.61  26-Sep-1999  lukem * replace ifdefs against __SVR4 and __linux__ with DIRENT_MISSING_D_NAMLEN;
it's more portable and more obvious
* remove the mkgmtime() && HAVE_TIMEGM stuff:
a) why should netbsd have to define HAVE_TIMEGM to compile cleanly?
b) foreign compiles of ftp should just be linked with working
timegm function

a more portable version of this ftp client will be released as a 3rdparty
product; no use polluting our code with half-baked attempts...
 1.60  24-Sep-1999  lukem * use %lld instead of %qd to print out (long long) vars.
(slightly more portable; e.g, solaris supports this)
* remove some fluff (lint)
 1.59  24-Sep-1999  lukem * finish replacing snprintf() with sprintf(), for portability reasons.
add lots of comments about how to size up the buffers, and add extra
checks to hopefully ensure that there won't be an overflow (unless
someone modifies the length of the sprintf()s).
* as part of the above, slightly rework the way the `*' bar is calculated.
also fixes a display bug when > 160 stars were needed to be printed.
the maximum progress bar width at this time is 256.
* remove some code that checks the port that was #if 0-ed out as part of the
ipv6 migration; it's not going to be used again.
 1.58  22-Sep-1999  lukem branches: 1.58.2;
* add support for `xferbuf', which sets both `sndbuf' and `rcvbuf'
* document the above three commands
* rototill the way the sndbuf and rcvbuf work. remove resetsockbufsize()
* use the appropriate socket buffer size as the size of the buffer that
the read()/write() loops use. speeds up things in some cases.
 1.57  22-Sep-1999  lukem replace snprintf() with strlcpy(), strlcat(), or direct assignment
where appropriate. (strlc*() are easier to port to foriegn systems).

XXX; there's still a few snprintf's in the progress meter stuff to convert
 1.56  21-Sep-1999  lukem cast 4th arg to [gs]etsockopt() to (void *), to satisfy some foriegn systems
 1.55  12-Jul-1999  lukem * change fetch_ftp() to be fully rfc 1738 compliant; if the URL contains
the an empty directory (e.g, between `some' and `path' in
`ftp://host/some//path'), then execute `CWD ' (without a path).
This command will probably fail on rfc 959 compliant servers, so
issue a warning in this case and bail. [noted by cgd].
(i wonder if the people who wrote rfc 1738 actually realised that this
requirement appears to contravene the spec for `cwd' in rfc 959 ?)
* replace isurl() with isipv6addr(), and use appropriately. fixes
auto-login with `classic ftp URLs' (e.g, `ftp somehost:')
* cleanup and rework some of the ipv6 stuff in parse_url()
* prevent potential coredump in fetch_ftp() when parsing `;type=X'
* KNF a few lines
* fix a couple of comments
* cleanup the man page a bit
 1.54  02-Jul-1999  itojun add dual-stack (IPv4/v6) support. hope I broke no other part...
 1.53  29-Jun-1999  lukem [fear this; more ftp hacking from lukem :-]

features:
 1.52  26-Jun-1999  lukem always compile in ptransfer() (and bits of progressmeter()), so that
transfer stats are printed when verbose is enabled even if ftp is compiled
with -DNO_PROGRESS/-DSMALL. this adds back some feedback that existed
in traditional ftp that was lost when i added the progress meter. (noticed
when ftp-ing from the -DSMALL version of ftp that's on the 1.4 boot floppy)
 1.51  24-Jun-1999  christos Avoid free'ing junk. It pisses off gnumalloc.
 1.50  20-Jun-1999  cgd split the "SMALL" #ifdefs into ones for NO_ABOUT, NO_EDITCOMPLETE, and
NO_PROGRESS. -DSMALL still implies all of those. progress meter support
isn't necessary for the smallest possible ftp client, but it adds very
little space and makes users' lives much better. Therefore, it should
be enabled for installation media if at all possible.
 1.49  02-Jun-1999  lukem * fix gate mode to login as `user@realhost' rather than using PASSERVE;
the latter only seemed to work for TIS Gauntlet and not TIS fwtk.
thanks to simonb@netbsd.org for testing this. fixes [bin/5556].
* if EOF (e.g, ^D) is entered at a username/password/account prompt which
happens to use fgets(), exit the login rather than treating EOF as CR.
* don't use the comma operator where separate statements are valid
* always use snprintf to copy stuff into malloced buffers, just in case
typos creep in and mean that the buffer ends up being overflowed
 1.48  20-May-1999  matthias ftp_login would only return success for proxy connections after the last
change. Fix this by setting rval at the right place.
 1.47  05-May-1999  lukem * fix ftp_login() so that -a always logs in anonymously, even if a
.netrc entry exists for the host with a valid user. noted by
Frederick Bruckman <fb@enteract.com> in [bin/7477]
* fix a minor memory leak associated with calling ruserpass() with
an empty user, password, or account.
 1.46  08-Mar-1999  lukem branches: 1.46.2;
whitespace & copyright fixes
 1.45  12-Feb-1999  lukem always run progressmeter() even if it's not the foregroundproc(), as some
calculations need to occur to prevent the calculated xfer rate from being
too fast if ftp(1) is backgrounded for a while during the xfer.
bug found by thor lancelot simon <tls@netbsd.org>.
 1.44  07-Feb-1999  lukem ftpvis(): escape LF too
 1.43  07-Feb-1999  lukem changes to fix [bin/6951] by Peter Simons <simons@cys.de>:
* implement ftpvis(), which \ quotes the following in the given string:
SP, TAB, \, CR, "
* use ftpvis() in complete_ambiguous(), to escape characters in a word
which would confuse makeargv()/slurpstring().
 1.42  31-Jan-1999  lukem revert previous write->fwrite conversion in progressmeter(); stdio is bad in signal handlers, nkay?
 1.41  24-Jan-1999  lukem * -v enables verbose & progress, -V disables both
* set setvbuf(ttyout, NULL, _IOLBF, 0) and remove a bunch of fflush(ttyout).
* use fwrite() instead of write() for progressmeter (don't intermix stdio
with non stdio ops)
 1.40  05-Jan-1999  lukem whitespace fix
 1.39  01-Jan-1999  lukem improve previous, so that display rounding is done to binary boundaries
(1024 not 1000) because the prefix determination is done that way.
tested on various boundary conditions.
 1.38  01-Jan-1999  lukem add two digits of precision to the transfer rate, and always show at
least KB/s. instead of printing `5414 B/s' now print `5.41 KB/s'.
 1.37  29-Dec-1998  lukem be consistent with return (foo) style
 1.36  18-Nov-1998  itohy Temporary fix of UTC -> time_t conversion:
1. changed to use timegm(3) on NetBSD and
2. supply alternative code for other Unix-like OSs
(NetBSD ftp shall be portable for some reasons :-).

This fix closes PR #6448.

NOTE: This should be fixed again if a portable UTC to time_t
conversion method is specified in some standards.
 1.35  06-Nov-1998  christos char -> unsigned char
 1.34  28-Sep-1998  lukem * add transfer rate to progress meter. (suggested by hubert feyrer)
* fix up progress meter display for files once > ~20.47MB has been
transferred. (noted by darren reed)
* use xsignal() for SIGWINCH, so window resizes don't abort xfers on
drainbamaged systems.
* don't assume that TIOCGWINSZ returns a non zero width.
 1.33  08-Aug-1998  lukem * implement xsignal(); same semantics as signal() but uses sigaction
with an explicit SA_RESTART. (needed for portability)
* use xsignal() for SIGALRM and SIGINFO handlers
 1.32  08-Aug-1998  lukem missed a NO_QUAD situation. be consistent in use of ifndef NO_QUAD
 1.31  03-Aug-1998  lukem features:
* support $no_proxy, which is a comma or space separated list of
host[:port] elements for which proxying is to be disabled.
(asked for by cgd in [bin/5027])
* if $FTPANONPASS is defined, use that as the anon ftp password
(instead of "`whoami`@")
* allow http URL's without a filename as long as an output file
is specified.

other stuff:
* implement parse_url(), which breaks up a URL into its bits, and use.
* simplify url_get() and auto_fetch() to use parse_url() and to not
modify the supplied URL or a copy of it.
* implement xmalloc() and xstrdup(); error-checked malloc()/strdup()
* add more consistency to messages, quoting strings in output as `%s'
 1.30  29-Jul-1998  lukem check EL_EDITMODE after el_source and disable editing if necessary
 1.29  26-Jul-1998  mycroft const poisoning.
 1.28  22-Jul-1998  lukem * send 'Connection: close' in HTTP/1.1 headers, preventing the
remove server from using a persistent connection, which speeds
up such requests.
* support http 301 and 302 redirects
* rewrite guts of url_get() to use fparseln() et al instead of
read(s,&p,1)... enables each in the header to be parsed
as necessary
* rename login to ftp_login, to remove conflict with util.h::login
* cleanup verbose messages during http proxy requests
 1.27  10-Jul-1998  thorpej Add "sndbuf" and "rcvbuf" commands for setting the socket buffer sizes,
which in turn can allow the use of larger TCP windows. This is a work in
progress; there is not yet support for specifying global defaults or
user prefrences on a host/network basis.
 1.26  06-Jul-1998  mrg - use an array MAXHOSTNAMELEN+1 size to hold hostnames
- ensure hostname from gethostname() is nul-terminated in all cases
- minor KNF
- use MAXHOSTNAMELEN over various other values/defines
- be safe will buffers that hold hostnames
 1.25  19-Jun-1998  kleink GLOB_QUOTE is dead; per POSIX, backslash quoting of special characters being
enabled is the default behaviour.
 1.24  04-Jun-1998  lukem some fixes & enhancements from openbsd's ftp, with extra fixes by me:
* default to passive with active fallback. $FTPMODE modifies this behaviour.
-A forces active connection.
* support '-o outfile' for auto-fetched files. outfile can be a file,
`-' (for stdout), or '|command' (to output each file through command).
* support '-r waittime', which retries the connection after waittime seconds
if it fails.
* fix 'page file' when restart is non-zero.
* try all ip-addresses of a host in a http fetch (as the normal ftp fetch
does).

XXX: a ``broken pipe'' error sometimes occurs with -o '|command';
i haven't tracked this down yet.
 1.23  20-May-1998  christos - add <signal.h> since we are using signals.
- cast arg to is*() to unsigned char.
- don't require quad_t to exist to compile.
- adapt to the new libedit version.
 1.22  04-Feb-1998  christos Don't hardcode 1900
 1.21  20-Jan-1998  lukem cosmetic mods to ptransfer() (which displays transfer statistics for
SIGINFO and at the end of the transfer):
* display time taken as [[x days?] hh:]mm:ss
* display ETA as [hh:]mm:ss
* print ' (stalled)' if the transfer is stalled
 1.20  18-Jan-1998  lukem define MAX_IN_PORT_T and use instead of USHRT_MAX. suggested by cgd@netbsd.org
 1.19  18-Jan-1998  lukem * ensure buffer for username is initialised, so ^D on username prompt
doesn't use garbage for the username. from "Soren S. Jorvang" <soren@t.dk>
in [bin/4559]
* use in_port_t for ports, and USHRT_MAX instead of 0xffff
(from millert@openbsd.org)
* use `NULL' instead of `(.... *)0' where appropriate.
 1.18  12-Dec-1997  gwr Kill the "progress bar" stuff when compiled with -DSMALL
because progressmeter() uses floats which are unwanted
when this is built under src/distrib/utils/x_ftp
(Maybe -DNO_FLOAT would be better?)
 1.17  01-Nov-1997  lukem * in recvrequest(), ignore restart_point unless "RETR"ieving. fixes problems
where a remote completion or `mget' would confuse the client a `restart'
had been issued beforehand. now, `restart' is remembered until an operation
that can actually use it is invoked.
* in sendrequest(), don't reset restart_point upon entry. fixes `restart'
for `put' operations.
* if `restart' is invoked with no arguments, print current setting instead
of displaying a usage
* consistently use printf("%qd", (long long)restart_point) when displaying
restart_point
* use strto[lq]() instead of atol() when parsing `mark' and `restart' values
* remove unnecessary strlen()s when result of previous snprintf() will do
* replace a few malloc()/strcpy()s with strdup()s
* use SECSPERHOUR instead of '3600'
 1.16  14-Oct-1997  christos branches: 1.16.2;
PR/4257: Jaromir Dolecek: Compensate for the libedit changes.
 1.15  11-Oct-1997  mycroft If the ETA is too large to fit on the display (i.e. 100 hours or more), don't
try to display it.
Also, eliminate many extra calls to strlen().
 1.14  21-Sep-1997  lukem More cleanups from Todd Miller <Todd.Miller@courtesan.com>:
* Use an int, not "union wait".
* Move the "parsed_url" label so that the next statement is not an "else" --
some ansi C compilers don't like it the old way (SGI's for example).
* Deal with the possibility of getlogin(2) failing.
* Don't error out if the remote server doesn't support the "MDTM" command.
 1.13  23-Aug-1997  lukem support $TMPDIR
 1.12  18-Aug-1997  lukem bugs fixed:
* don't interpret '-' or '|' when a local filename is determined from
the remote name (i.e, in mget, and in get with only one argument).
This is implemented using an extra argument to recvrequest().
Fixes a major security hole.
* clean up memory leak when using globulize()
* clean up a couple of comments
* fix wording in TNF copyright

features added:
* support for TIS fwtk gate-ftp servers:
* read defaults from $FTPSERVER && $FTPSERVERPORT
* start in gate-ftp mode if invoked as 'gate-ftp'
* toggle or set with 'gate [host [port]]'
 1.11  21-Jul-1997  lukem fix compile on alpha:
* cast all %qd printfs to (long long) [inspired by billc@warped.net]
* replace sscanf %qd with strtoq() et al
 1.10  20-Jul-1997  lukem * use RCSID() && COPYRIGHT() macros
* cleanup for WARNS=1 (including some ugly '(void)&var' bits wrapped in
#ifdef __GNUC__ to shut up gcc warnings WRT setjmp/longjmp)
* use strtol() instead of atoi(), and more extensively check result of
conversion
* use u_int16_t instead of short or int for TCP port addresses
 1.9  10-Jun-1997  lukem * Only print the progress bar when ftp(1) is the foreground progress.
Patch from enami tsugutomo <enami@ba2.so-net.or.jp> in [bin/3735].
* Fix punctuation.
 1.8  12-May-1997  lukem don't print -ve ETA if file size has grown. noticed by hpeyerl@netbsd.org
 1.7  14-Apr-1997  lukem More enhancements/bugfixes (when will it end?)
* differentiate between being connected, and being logged in
* cleanup some text messages
* support username & password ftp URLs (ftp://user:pass@host/) in non-proxy
situations; assume proxy supports it for proxy situations.
* cd to / before performing any autofetch transfers
* use strncasecmp in URL parsing. fix from <Todd.Miller@courtesan.com>
 1.6  05-Apr-1997  lukem * support $ftp_proxy for ftp:// transfers [bin/3245]
* add "more" & "less" as synonyms for "page"
* move editline setup code into controlediting(), and call appropriately.
only setup setup terminal if going into interactive mode. inspired
by Todd Miller <todd.miller@courtesan.com>
 1.5  13-Mar-1997  lukem Features:
* support remglobbing of auto_fetch arguments
* new flag - '-e'; disable editing
* "page file" == "get file |${PAGER-less}"

Bugfixes/cleanup:
* consistently use a trailing '.' on messages
* code cleanup, including buffer overrun fixes, use puts
and putchar in places, etc (inspired by OpenBSD mods)
* disable progress bar when local-file is a pipe or '-'
* skip \r in http headers
* fix remote ftpd slash bug more elegantly (so it works with ////)
* abort_remote(): check if cout==NULL before using it. should fix [bin/3273]
* fixed up cosmetic problems when complete_remote() generated errors from the
remote server (such as "no files found", "login with user and pass", ...)
done by adding extra argument to remglob(), which is a pointer to an error
buffer to put messages in rather than printing to stdout.
 1.4  01-Feb-1997  lukem don't display negative ETA upon SIGINFO because filesize isn't known
 1.3  01-Feb-1997  lukem [Yet Another Huge Ftp Commit - hopefully the last for a while,
barring any more little things people want added ...]

New features:
* progressmeter is now asynchronous, so "stalled" transfers can be
detected. "- stalled -" is displayed instead of the ETA in this case.
When the xfer resumes, the time that the xfer was stalled for is
factored out of the ETA. It is debatable whether this is better than
not factoring it out, but I like it this way (I.e, if it stalls for 8
seconds and the ETA was 30 seconds, when it resumes the ETA will still
be 30 seconds).
* verbosity can be disabled on the command line (-V), so that in auto-fetch
mode the only lines displayed will be a description of the file, and
the progress bar (if possible)
* if the screen is resized (and detected via the SIGWINCH signal), the
progress bar will rescale automatically.

Bugs fixed:
* progress bar will not use the last character on the line, as this can
cause problems on some terminals
* screen dimensions (via ioctl(TIOCWINSZ)) should use stdout not stdin
* progressmeter() used some vars before initialising them
* ^D will quit now. [fixes bin/3162]
* use hstrerror() to generate error message for host name lookup failure.
* use getcwd instead of getwd (it should have been OK, but why tempt fate?)
* auto-fetch transfers will always return a positive exit value upon failure
or interruption, relative to the file's position in argv[].
* remote completion of / will work, without putting a leading "///".
This is actually a bug in ftpd(1), where "NLST /" prefixes all names
with "//", but fixing every ftpd(1) is not an option...
 1.2  30-Jan-1997  thorpej Don't try to divide by zero in progressmeter().
From Thorsten Frueauf <frueauf@ira.uka.de>, PR #3160.
 1.1  19-Jan-1997  lukem New features:
* Command line editing via editline(3) library.
* Context sensitive command and file completion, including remote files.

Enhancements to auto-fetch feature:
* Support for http:// URLs using the http protocol, including proxy HTTP
support via $htty_proxy if it's defined.
* The connection is kept open between successive files on the same host.
(obviously, this does not count for http requests.)
* Return value of ftp is 0 on no error, or the offset in argv[] of the file
which failed (i.e., argv[x] failed, ftp returns x).
* If the path in an ftp URL or classic format line has a trailing '/',
cd to the path and enter interactive mode. Fixes [bin/3011], albiet
requiring the user to help ftp in determining the operation.

Other changes:
* '-P port' works for normal ftp, and is the default for all classic style
auto-fetch transfers and for ftp URLs that don't specify the port.
(previously it would just work for the first xfer.)
* Some code moved into separated files along logical divisions.
* Editing and completion can be compiled out with -DSMALLFTP.
 1.16.2.6  18-Jan-1999  cgd patch to fix 1.3.3's filename completion problems. Fixes PR#6647. (frueauf)
 1.16.2.5  23-Nov-1998  cgd pull up rev 1.36 from trunk (itohy)
 1.16.2.4  15-Nov-1998  cgd patch from Simon Burge to make this actually compile on the branch
 1.16.2.3  10-Nov-1998  cgd pull up rev(s) 1.19-1.35 from trunk and patch to work on 1.3 branch. (feyrer)
 1.16.2.2  14-Dec-1997  mellon Pull rev 1.18 up from trunk (gwr)
 1.16.2.1  18-Nov-1997  mellon Pull rev 1.17 up from trunk (lukem)
 1.46.2.3  12-Feb-2000  he Pull up revisions 1.87-1.90 (via patch, requested by he):
Work around lame ftp servers returning year 19100 for year 2000.
Fixes PR#9289.
 1.46.2.2  25-Jun-1999  cgd pull up rev 1.50 from trunk. (cgd)
 1.46.2.1  22-Jun-1999  perry pullup 1.46->1.49 (lukem)
 1.58.2.1  27-Dec-1999  wrstuden Pull up to last week's -current.
 1.95.2.1  23-Jun-2000  minoura Sync w/ netbsd-1-5-base.
 1.106.2.1  10-Feb-2003  jmc Pullup rev 1.110 (requested by grant in ticket #1115)
Separate progress reporting into a utility set of routines. Use
these to provide a standalone progress binary and integrate support
for this into sysinst.
 1.117.2.15  26-May-2006  ghen Pull up following revision(s) (requested by jnemeth in ticket #1339):
usr.bin/ftp/util.c: revision 1.135
Coverity CID 3610: derefencing NULL pointer.
 1.117.2.14  24-Jul-2005  tron branches: 1.117.2.14.2;
Pull up revision 1.131 (requested by lukem in ticket #606):
forgot to remove the file pointer from the macro.
 1.117.2.13  24-Jul-2005  tron Pull up revision 1.130 (requested by lukem in ticket #606):
Add NO_USAGE and NO_DEBUG so that we can fit in the floppies again.
 1.117.2.12  24-Jul-2005  tron Pull up revision 1.129 (requested by lukem in ticket #606):
Implement:
int getline(FILE *stream, char *buf, size_t buflen, const char **errormsg)
Read a line from the FILE stream into buf/buflen using fgets(), so up
to buflen-1 chars will be read and the result will be NUL terminated.
If the line has a trailing newline it will be removed.
If the line is too long, excess characters will be read until
newline/EOF/error.
Various -ve return values indicate different errors, and errormsg
will be changed to an error description if it's not NULL.
Convert to use getline() instead of fgets() whenever reading user input
to ensure that an overly long input line doesn't leave excess characters
for the next input operation to accidentally use as input.
Zero out the password & account after we've finished with it.
Consistently use getpass(3) (i.e, character echo suppressed) when
reading the account data. For some reason, historically the "login"
code suppressed echo for Account: yet the "user" command did not!
Display the hostname in the "getaddrinfo failed" warning.
Appease some -Wcast-qual warnings. Fixing all of these requires
significant code refactoring. (mmm, legacy code).
 1.117.2.11  24-Jul-2005  tron Pull up revision 1.128 (requested by lukem in ticket #606):
formatbuf(): fix %m and %M to use the hostname, not the username.
 1.117.2.10  24-Jul-2005  tron Pull up revision 1.127 (requested by lukem in ticket #606):
Improve method used in fileindir() to determine if `file' is in or under `dir':
realpath(3) on non-NetBSD systems may fail if the target filename doesn't
exist, so instead use realpath(3) on the parent directory of `file'.
Per discussion with Todd Eigenschink.
 1.117.2.9  24-Jul-2005  tron Pull up revision 1.126 (requested by lukem in ticket #606):
tab cleanup
 1.117.2.8  24-Jul-2005  tron Pull up revision 1.125 (requested by lukem in ticket #606):
Use size_t instead of int where appropriate.
 1.117.2.7  24-Jul-2005  tron Pull up revision 1.124 (requested by lukem in ticket #606):
Some const cleanups.
 1.117.2.6  24-Jul-2005  tron Pull up revision 1.123 (requested by lukem in ticket #606):
Helps if the definition of xconnect() matches its declaration....
 1.117.2.5  18-May-2005  snj Pull up revision 1.122 (requested by lukem in ticket #301):
* Correct the "optlen" argument passed to getsockopt(3) and setsockopt(3)
in various places. Fixes a problem noted by Allen Briggs.
* Improve warning printed when connect(2) for the data channel fails.
 1.117.2.4  18-May-2005  snj Pull up revision 1.121 (requested by lukem in ticket #318):
Use socklen_t instead of int as the 5th argument to getsockopt().
Improve invocation of setsockopt() and associated failure messages.
 1.117.2.3  09-May-2005  tron Pull up revision 1.120 (requested by lukem in ticket #266):
correct a comment
 1.117.2.2  09-May-2005  tron Pull up revision 1.119 (requested by lukem in ticket #266):
gratuitous whitespace cleanup (before someone else jumps the gun...)
 1.117.2.1  09-May-2005  tron Pull up revision 1.118 (requested by lukem in ticket #265):
Implement a timeout on the accept(2) in dataconn() and the
connect(2) in xconnect() by temporarily setting O_NONBLOCK
on the socket and using xpoll() to wait for the operation
to succeed.
The timeout used is the '-q quittime' argument (defaults to
60s for accept(2), and the system default for connect(2)).
Idea inspired by discussion with Chuck Cranor.
This may (indirectly) fix various problems with timeouts
in active mode through broken firewalls.
Implement xpoll() as a wrapper around poll(2), to make it
easier to replace on systems without a functional poll(2).
Unconditionally use xpoll() instead of conditionally using
select(2) or poll(2).
 1.117.2.14.2.1  26-May-2006  ghen Pull up following revision(s) (requested by jnemeth in ticket #1339):
usr.bin/ftp/util.c: revision 1.135
Coverity CID 3610: derefencing NULL pointer.
 1.134.2.1  19-Jun-2006  chap Sync with head.
 1.143.4.1  09-Jan-2008  matt sync with HEAD
 1.144.6.1  18-May-2008  yamt sync with head.
 1.146.2.2  18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.146.2.1  23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.148.6.1  13-May-2009  jym Sync with HEAD.

Third (and last) commit. See http://mail-index.netbsd.org/source-changes/2009/05/13/msg221222.html
 1.148.4.1  20-Nov-2010  riz Pull up following revision(s) (requested by lukem in ticket #1370):
usr.bin/ftp/extern.h: revision 1.78
usr.bin/ftp/fetch.c: revision 1.192
usr.bin/ftp/util.c: revision 1.153
Parse HTTP 'Date' entries in the `C' locale rather than the user's.
Fix from [bin/42917] (with minor changes), from KAMADA Ken'ichi.
 1.155.6.3  22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.155.6.2  30-Oct-2012  yamt sync with head
 1.155.6.1  17-Apr-2012  yamt sync with head
 1.156.2.1  17-Dec-2013  bouyer Apply patch, requested by tron in ticket #997:
usr.bin/ftp/Makefile patch
usr.bin/ftp/cmds.c patch
usr.bin/ftp/cmdtab.c patch
usr.bin/ftp/extern.h patch
usr.bin/ftp/fetch.c patch
usr.bin/ftp/ftp.1 patch
usr.bin/ftp/ftp.c patch
usr.bin/ftp/ftp_var.h patch
usr.bin/ftp/main.c patch
usr.bin/ftp/progressbar.c patch
usr.bin/ftp/ssl.c patch
usr.bin/ftp/ssl.h patch
usr.bin/ftp/util.c patch
usr.bin/ftp/version.h patch

Add HTTPS support to ftp(1).
 1.157.2.1  25-Feb-2013  tls resync with head
 1.158.22.3  12-Sep-2022  martin Catch up to current, requested by christos in ticket #1763:

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/cmds.c up to 1.141
usr.bin/ftp/complete.c up to 1.47
usr.bin/ftp/domacro.c up to 1.23
usr.bin/ftp/extern.h up to 1.82
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp.c up to 1.174
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/progressbar.c up to 1.24
usr.bin/ftp/progressbar.h up to 1.9
usr.bin/ftp/ssl.c up to 1.12
usr.bin/ftp/ssl.h up to 1.5
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

ftp(1): validate address from PASV and LPSV response.
ftp(1): use raw write(2) instead of fwrite(3) to avoid stream
corruption because of the progress bar interrupts.
Fixes for PR 56219 and PR 55857.
PR 57003: Support relative redirects.
 1.158.22.2  12-Sep-2022  martin Backout ticket #1763 for now - trust anchors are not solved.
 1.158.22.1  12-Sep-2022  martin Catch up to current, requested by christos in ticket #1763:

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/cmds.c up to 1.141
usr.bin/ftp/complete.c up to 1.47
usr.bin/ftp/domacro.c up to 1.23
usr.bin/ftp/extern.h up to 1.82
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp.c up to 1.174
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/progressbar.c up to 1.24
usr.bin/ftp/progressbar.h up to 1.9
usr.bin/ftp/ssl.c up to 1.11
usr.bin/ftp/ssl.h up to 1.5
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

ftp(1): validate address from PASV and LPSV response.
ftp(1): use raw write(2) instead of fwrite(3) to avoid stream
corruption because of the progress bar interrupts.
Fixes for PR 56219 and PR 55857.
PR 57003: Support relative redirects.
 1.159.4.1  13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.160.2.4  12-Sep-2022  martin Catch up to current, requested by christos in ticket #1523

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/ssl.c up to 1.12
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

PR 57003: Support relative redirects.
 1.160.2.3  12-Sep-2022  martin Back out ticket #1523 for now - trust anchor validation is not solved.
 1.160.2.2  12-Sep-2022  martin Catch up to current, requested by christos in ticket #1523

usr.bin/ftp/Makefile up to 1.39
usr.bin/ftp/fetch.c up to 1.235
usr.bin/ftp/ftp.1 up to 1.147
usr.bin/ftp/ftp_var.h up to 1.86
usr.bin/ftp/main.c up to 1.128
usr.bin/ftp/ssl.c up to 1.11
usr.bin/ftp/util.c up to 1.164
usr.bin/ftp/version.h up to 1.94

PR 57003: Support relative redirects.
 1.160.2.1  14-Jun-2021  martin Pull up following revision(s) (requested by lukem in ticket #1291):

usr.bin/ftp/ftp.c: revision 1.169
usr.bin/ftp/util.c: revision 1.161

ftp: exit if lostpeer invoked by a signal

lostpeer() calls too many async-unsafe functions (both directly
and indirectly) to close and cleanup the remote connections,
so just exit after the cleanup if invoked by a signal.

Reported in private mail by Qi Hou.

May also resolve a crash reported by Thomas Klausner.
 1.164.2.4  02-Dec-2024  martin Pull up following revision(s) (requested by lukem in ticket #1021):

usr.bin/ftp/ftp.c: revision 1.178
usr.bin/ftp/version.h: revision 1.99
usr.bin/ftp/ruserpass.c: revision 1.35
usr.bin/ftp/main.c: revision 1.134
usr.bin/ftp/main.c: revision 1.135
usr.bin/ftp/progressbar.c: revision 1.27
usr.bin/ftp/util.c: revision 1.165
usr.bin/ftp/cmds.c: revision 1.144
usr.bin/ftp/extern.h: revision 1.84
usr.bin/ftp/fetch.c: revision 1.242
usr.bin/ftp/ftp.1: revision 1.160

s/bninary/binary/ in comment.

extract duplicate code into a function.

Check bounds when copying to destination.

Remove const where the const string ended up being overwritten.

use unsigned when doing shifts.

remove const when string gets overwritten.

ftp: exit non-zero if short http transfer when filesize is known
If a http file size is known and the fetch finishes with less bytes
transferred, exit non-zero.
Bump version to 20241129.
PR bin/54713
PR bin/58281

ftp: help improvements
Document -? as a separate mode.
Document -H HEADER in the usage.
Clarify units for -b and -x.
Consistent argument names between ftp -? and ftp(1).

ftp: order getopt Upper before lower
Consistently order options in getopt and the switch
with the upper case option before the lower case option.
This makes it easier to cross-reference with -? and ftp(1).
No functional change.
 1.164.2.3  13-Oct-2024  martin Pull up following revision(s) (requested by riastradh in ticket #970):

tests/usr.bin/Makefile: revision 1.42
usr.bin/ftp/ruserpass.c: revision 1.34
usr.bin/ftp/main.c: revision 1.130
usr.bin/ftp/ssl.c: revision 1.17
usr.bin/ftp/main.c: revision 1.131
usr.bin/ftp/ssl.c: revision 1.18
usr.bin/ftp/main.c: revision 1.132
usr.bin/ftp/ssl.c: revision 1.19
usr.bin/ftp/main.c: revision 1.133
distrib/sets/lists/tests/mi: revision 1.1342
usr.bin/ftp/ftp.1: revision 1.151
usr.bin/ftp/ftp.1: revision 1.152
usr.bin/ftp/progressbar.c: revision 1.25
usr.bin/ftp/ftp.1: revision 1.153
usr.bin/ftp/progressbar.c: revision 1.26
usr.bin/ftp/ftp.1: revision 1.155
usr.bin/ftp/ftp.1: revision 1.156
usr.bin/ftp/fetch.c: revision 1.239
usr.bin/ftp/ftp.1: revision 1.157
usr.bin/ftp/ftp.1: revision 1.158
usr.bin/ftp/ftp.1: revision 1.159
usr.bin/ftp/ftp_var.h: revision 1.87
etc/mtree/NetBSD.dist.tests: revision 1.208
usr.bin/ftp/ftp_var.h: revision 1.88
usr.bin/ftp/ftp_var.h: revision 1.89
usr.bin/ftp/cmds.c: revision 1.142
usr.bin/ftp/util.c: revision 1.168
usr.bin/ftp/cmds.c: revision 1.143
tests/usr.bin/ftp/custom_headers.sh: revision 1.1
usr.bin/ftp/ssl.c: revision 1.20
usr.bin/ftp/complete.c: revision 1.48
tests/usr.bin/ftp/Makefile: revision 1.1
tests/usr.bin/ftp/t_custom_headers.sh: revision 1.1
usr.bin/ftp/fetch.c: revision 1.240
usr.bin/ftp/fetch.c: revision 1.241
usr.bin/ftp/ftp.c: revision 1.176
usr.bin/ftp/ftp.c: revision 1.177
(all via patch)

ftp(1): wording and formatting improvements

Fix grammar issue with "Support values" reported in private mail.
Document all file transfer types in "type" and cross-reference that.
Consistency fixes in describing file transfer parameters and types.

Fix some mandoc -Tlint issues (except "useless macro: Tn").

Add -b <buflen> to specify the buffer size.

ftp: bump FTPBUFLEN from 4kB to 16kB
sourceforge.net returns a 5kB content-security-policy.
Analyzed by mlelstv@ who reports usual limits are between 4kB and 48kB.
default is now 16K

ftp: improve -b documentation

Order -b bufsize in the synopsis.

Document the actual default value.

ftp: improve units used in comments and errors
Use "KiB" instead of "K" in errors.
Clarify related comments.

pass some lint.

PR/58581: Sunil Nimmagadda: Add flag to allow specifying extra http header
fields.

ftp(1): Nix trailing whitespace in man page.
No functional change intended.

PR bin/58581: ftp(1) should allow specifying header fields in http requests
fix markup (h -> H), explain about multiple headers, fix usage (from RVP)

Don't forget the dot, use the intended macro name (I think),
and improve the wording a little. (All related to the -H option.)
ftp(1): Add test for custom HTTP header fields.

Based on a patch from Sunil Nimmagadda.

PR bin/58581: ftp(1) should allow specifying header fields in http
requests
 1.164.2.2  16-May-2023  martin Pull up following revision(s) (requested by lukem in ticket #171):

usr.bin/ftp/ssl.c: revision 1.15
usr.bin/ftp/util.c: revision 1.167
usr.bin/ftp/ftp.c: revision 1.175
usr.bin/ftp/version.h: revision 1.97

add timeout for ssl connect

Implement a timeout for SSL connection setup, using -q QUITTIME,
defaulting to 60 seconds.

SSL_connect(3) (unlike connect(2)) doesn't timeout by default.
Adapt ssl error messages destination: if unexpected error
from local API, use warn()/warnx() to stderr;
if expected error from a network operation (e.g., timeouts),
use fprintf to ttyout (which might be stdout).

Consistently use ftp_poll() instead of select();
ssl.c (using select()) was added 7 years after the
previous uses of select() were converted to poll().

Check EAGAIN as well as existing EINTR error from ftp_poll(),
for portability.
 1.164.2.1  16-May-2023  martin Pull up following revision(s) (requested by lukem in ticket #170):

usr.bin/ftp/ssl.c: revision 1.13
usr.bin/ftp/fetch.c: revision 1.236
usr.bin/ftp/util.c: revision 1.166
usr.bin/ftp/main.c: revision 1.129
usr.bin/ftp/extern.h: revision 1.83
usr.bin/ftp/ftp.1: revision 1.148
usr.bin/ftp/cmdtab.c: revision 1.53
usr.bin/ftp/version.h: revision 1.96

Add option sslnoverify to control validation of SSL certificates.

Add netrc processing to fetch-mode (URL on command line) to enable options and autologin
via netrc.

Fix SSL cleanup in some error paths.

Certificate validation is now enabled by default. Set FTPSSLNOVERIFY=1 in environment
or configure a corresponding init macro via netrc to not validate certs (required if
you haven't installed a required CA certificate for OpenSSL).

Discussed with lukem@ on icb.
 1.167.2.1  02-Aug-2025  perseant Sync with HEAD

RSS XML Feed