Home | History | Annotate | only in /src/usr.sbin/puffs/mount_psshfs
History log of /src/usr.sbin/puffs/mount_psshfs
RevisionDateAuthorComments
 1.4 22-Apr-2009  lukem Enable WARNS=4 by default, except for:
cpuctl dumplfs hprop ipf iprop-log kadmin kcm kdc kdigest
kimpersonate kstash ktutil makefs ndbootd ntp pppd quot
racoon racoonctl rtadvd sntp sup tcpdchk tcpdmatch tcpdump
traceroute traceroute6 user veriexecgen wsmoused zic
(Mostly third-party applications)
 1.3 18-Jul-2007  pooka branches: 1.3.16; 1.3.20;
get rid of DBG=
 1.2 07-Jan-2007  pooka man page for mount_psshfs
 1.1 29-Dec-2006  pooka puffs sshfs, an sshfs implementation built on top of the puffs
continuation framework
 1.3.20.1 13-May-2009  jym Sync with HEAD.

Third (and last) commit. See http://mail-index.netbsd.org/source-changes/2009/05/13/msg221222.html
 1.3.16.2 18-Jul-2007  pooka get rid of DBG=
 1.3.16.1 18-Jul-2007  pooka file Makefile was added on branch matt-mips64 on 2007-07-18 22:39:00 +0000
 1.25 23-Sep-2019  christos Restore binary compatibility by using the statvfs90 structure internally.
 1.24 22-Jun-2011  mrg branches: 1.24.42;
remove some always true if () checks GCC 4.5.3 picks up.
 1.23 29-Oct-2010  pooka Make error message more userfriendly in cases where server does
not support sftp.
 1.22 01-Apr-2010  pooka In case we create a file, reclaim the vnode, and lookup the file
without the directory timeout expiring, we get vattr_null as the
attributes for that file. Ensure that we always report sane
attributes by getting them from the server if this is the case.
(also, sprinkle some const)

Fixes problem reported by dyoung. But wait! The bug's medallion
begins to glow! The bug looks much better! I crumble to dust.

There's probably another similar bug related to "lazy open". It
will trigger if we reclaim a node before the response to the open
arrives. Even the comments (typoless) know about this bug ...
But verifying it exists and fixing it will have to wait for another
day.
 1.21 03-Feb-2010  pooka statvfs:

Since bresvd is not available from the server, calculate:
bresvd = bfree - bavail.

Then df calculates:
bavail = bfree - bresvd;

And we now get a much more accurate report from df on how much one
can write to the fs.
 1.20 20-May-2009  pooka Copyright year management. no functional change (unless sarah disagrees)
 1.19 20-May-2009  pooka Introduce -c [12], which can be used to open 1 or 2 ssh connections
to the server. If "2" is specified, a separate connection is used
for data and directory operations. Using two connections can
significantly increase directory operation performance on a saturated
link, at least up to 30x faster.
 1.18 23-Feb-2009  pooka Support statvfs@openssh.com extension where available. Makes df(1)
return something other than just 0.
caveat: statvfs is done for the mountpoint path, so might not give
the truth about a directory inside the mountpoint.
 1.17 06-Sep-2008  pooka branches: 1.17.6;
Due to a recent encounter with a sucky internet connection, add
experimental option -p, which tries to reestablish the connection
to the sftp server in case it is lost. This currently has a few
limitations (found in the man page), but generally works in some
use cases.

Better support might eventually emerge, but since that requires a
plunge into the depths of puffs_framebuf, I need quite a bit of
Fernet Branca to build up my courage before attempting it.
 1.16 12-Aug-2008  pooka void * -> puffs_cookie_t
no functional change
 1.15 30-Nov-2007  pooka branches: 1.15.8;
Rototill.

Ok, ok, a few more words about it: stop holding puffs_cc as a holy
value and passing it around to almost every possible place (popquiz:
which kernel variable does this remind you of?). Instead, pass
the natural choice, puffs_usermount, and fetch puffs_cc via
puffs_cc_getcc() only in routines which actually need it. This
not only simplifies code, but (thanks to the introduction of
puffs_cc_getcc()) enables constructs which weren't previously sanely
possible, say layering as a curious example.

There's still a little to do on this front, but this was the major
fs interface blast.
 1.14 27-Nov-2007  pooka Remove "puffs_cid" from the puffs interface following l-removal
from the kernel vfs interfaces. puffs_cc_getcaller(pcc) can be
used now should the same information be desired.
 1.13 01-Jul-2007  pooka branches: 1.13.4;
adapt: **newnode etc. pointers -> struct puffs_newinfo
 1.12 01-Jul-2007  pooka adapt: pid -> const struct puffs_cid *
 1.11 06-Jun-2007  pooka Move puffs to a two clause license where it already isn't so. And
as agc pointed out, even files with the third clause were already
effectively two clause because of a slight bug in the language...
 1.10 18-May-2007  pooka * use NOREF setback to prompt immediate (well, once the kernel
reference count goes to 0) reclaim of deleted nodes as opposed to
waiting for the system to start reclaiming the freelist
* combine some nodeflags from different variable to one status variable
 1.9 17-May-2007  pooka reflect changes in fs mount: use puffs_mount() instead of
puffs_domount() & puffs_start()
 1.8 05-May-2007  pooka Adapt to use puffs_framebuf. This gets rid of most of the duplicated
code between mount_psshfs and mount_9p and clarifies the code
structure.
 1.7 18-Apr-2007  pooka psbuf_recycle now exists, so use it instead of destroy&make in one
spot I probably missed originally. No real functional change.
 1.6 12-Apr-2007  pooka Actually, we can't reclaim our memory image for nodes which have
been nodetofh translated even if they are not valid on the sftp
server anymore, because some nfs client might still be clinging on
to the file handle we are reclaiming now.
 1.5 12-Apr-2007  pooka Support nfs exporting.

Now, when I say support, I mean "support", due to the limitations
of the backend. File handles are valid only for one session, since
nodes can only be identified by pathnames and pathnames don't (all)
fit into the nfs file handle space. Additionally, we can't detect
if a pathname is completely replaced by another file (if it's done
via some other route that through our mount, of course). But then
again, that's an inherent problem with sshfs even without nfs.
 1.4 12-Apr-2007  pooka Hide struct puffs_usermount from the rest of the world and provide
accessors for interesting data in it. Namely, you can now get
pu->pu_privdata with puffs_getspecific(), pu->pu_pn_root with
puffs_set/getroot() and pu->pu_maxreqlen with puffs_getmaxreqlen().
 1.3 15-Jan-2007  pooka * adapt to libpuffs path changes
* fix bug in rename
 1.2 06-Jan-2007  pooka * do mount as a forward operation instead of a callback
* process -o args for mount
 1.1 29-Dec-2006  pooka puffs sshfs, an sshfs implementation built on top of the puffs
continuation framework
 1.13.4.1 09-Jan-2008  matt sync with HEAD
 1.15.8.1 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.17.6.1 13-May-2009  jym Sync with HEAD.

Third (and last) commit. See http://mail-index.netbsd.org/source-changes/2009/05/13/msg221222.html
 1.24.42.1 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.27 25-Jun-2014  szptvlfn s/stavfs/statvfs/
 1.26 25-Dec-2012  reed branches: 1.26.6;
fix spelling typo
 1.25 15-Sep-2011  wiz branches: 1.25.2; 1.25.8;
Wording, from Snader_LB.
 1.24 03-Feb-2010  pooka Give credit: psshfs was inspired by FUSE sshfs.
 1.23 07-Jan-2010  wiz Don't be terse.
 1.22 07-Jan-2010  pooka Add -u and -g, which allow to remap one (local,remote) uidgid, i.e.
umapfs without the fuss.
 1.21 20-May-2009  pooka Copyright year management. no functional change (unless sarah disagrees)
 1.20 20-May-2009  pooka Introduce -c [12], which can be used to open 1 or 2 ssh connections
to the server. If "2" is specified, a separate connection is used
for data and directory operations. Using two connections can
significantly increase directory operation performance on a saturated
link, at least up to 30x faster.
 1.19 26-Feb-2009  wiz Make "sftp" a link.
 1.18 25-Feb-2009  pooka add caveat about info displayed by df(1)
 1.17 06-Sep-2008  pooka branches: 1.17.6;
Due to a recent encounter with a sucky internet connection, add
experimental option -p, which tries to reestablish the connection
to the sftp server in case it is lost. This currently has a few
limitations (found in the man page), but generally works in some
use cases.

Better support might eventually emerge, but since that requires a
plunge into the depths of puffs_framebuf, I need quite a bit of
Fernet Branca to build up my courage before attempting it.
 1.16 14-Dec-2007  jmmv branches: 1.16.6;
Add a -F option to allow passing a configuration file to ssh(1), instead
of having to use multiple -O options on the command line. OKed by pooka@.
 1.15 03-Dec-2007  wiz Do not overload Xr macro.
 1.14 02-Dec-2007  wiz Sort SEE ALSO. Minor nit fixes.
 1.13 08-Nov-2007  pooka Make it possible to force attribute/directory cache re-read by
sending SIGHUP to mount_psshfs.
 1.12 08-Nov-2007  pooka Make attribute & directory content caching timeout a command line
option. Use -t -1 to never expire, which is nifty on a high-latency
link where you know the server won't be modified from under you.
 1.11 20-Oct-2007  pooka Add option -r to specify maximum number of active read requests
for each node. Setting this to a small number can be used to
improve interactive performance on low-bandwidth links when performing
bulk data reads. Of course I could also open separate pipes for
bulk and other, but this was quicker and less intrusive and doesn't
require authenticating twice.
 1.10 24-Jun-2007  pooka branches: 1.10.4;
PUFFS_KFLAG_NOCACHE became two, so introduce the command line options
-o nonamecache and -o nopagecache and adapt file systems where necessary.
 1.9 09-May-2007  pooka remove caveat no longer applicable
 1.8 09-May-2007  tnn Add an EXAMPLES section.
 1.7 09-May-2007  tnn Change [-O sshopt value] to [-O sshopt=value] to better conform with the
way other userland apps behave. In /etc/fstab one would specify options
with -O=sshopt=value.
 1.6 09-May-2007  tnn Add flag [-O sshopt value] to mount_psshfs(8), allowing the user to specify
options to pass along to ssh(1). Remove redundant flag [-p sshport]; an
equivalent of -O Port sshport. ok'd by pooka@
 1.5 29-Apr-2007  pooka Add -p to specify server port if someone should for some reason
happen to run sshd on port, to pick a random number, 443.

Probably should give a way to specify rest of the ssh options.
 1.4 12-Apr-2007  pooka document -e for nfs exportability
 1.3 17-Feb-2007  pooka * document -s
* explicitly mention that this implements sshfs functionality
 1.2 08-Jan-2007  wiz New sentence, new line. Sort SEE ALSO. Punctuation nit.
 1.1 07-Jan-2007  pooka man page for mount_psshfs
 1.10.4.2 09-Jan-2008  matt sync with HEAD
 1.10.4.1 06-Nov-2007  matt sync with HEAD
 1.16.6.1 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.17.6.1 13-May-2009  jym Sync with HEAD.

Third (and last) commit. See http://mail-index.netbsd.org/source-changes/2009/05/13/msg221222.html
 1.25.8.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.25.8.1 25-Feb-2013  tls resync with head
 1.25.2.1 23-Jan-2013  yamt sync with head
 1.26.6.1 10-Aug-2014  tls Rebase.
 1.66 16-Sep-2021  andvar fix typos in word "successfully", mainly s/succesfully/successfully/.
 1.65 07-Oct-2016  christos PR/514612: Mateusz Paszwa: When no more data, bail out (reading from empty
files) instead of stuck looping.
 1.64 20-Jan-2012  jakllsch branches: 1.64.22;
Ensure psshfs_node_read() completely reads all data before return.
Fixes file corruption in psshfs exposed with kernel MAXPHYS at 128KiB.
 1.63 12-Aug-2011  riastradh branches: 1.63.2;
Cache vattr in psshfs's setattr.

This means within the cache window, a setattr that wouldn't change the
remote file's attributes from our current view of them will not be
relayed to the server and wait for the server to answer. Thus, e.g., a
process with a periodic timer interrupt that calls open(2) in a loop
can make progress with much higher probability than without caching.

XXX The test case doesn't work, so it's currently disabled. It needs
to stop the child of sshd that is handling an sftp session, not sshd
itself, and it's not obvious how to do that.

ok pooka
 1.62 29-Oct-2010  pooka Make error message more userfriendly in cases where server does
not support sftp.
 1.61 01-Apr-2010  pooka In case we create a file, reclaim the vnode, and lookup the file
without the directory timeout expiring, we get vattr_null as the
attributes for that file. Ensure that we always report sane
attributes by getting them from the server if this is the case.
(also, sprinkle some const)

Fixes problem reported by dyoung. But wait! The bug's medallion
begins to glow! The bug looks much better! I crumble to dust.

There's probably another similar bug related to "lazy open". It
will trigger if we reclaim a node before the response to the open
arrives. Even the comments (typoless) know about this bug ...
But verifying it exists and fixing it will have to wait for another
day.
 1.60 07-Jan-2010  pooka Add -u and -g, which allow to remap one (local,remote) uidgid, i.e.
umapfs without the fuss.
 1.59 05-Nov-2009  pooka more lp64 it's-a-sign! fixes
 1.58 20-May-2009  pooka Copyright year management. no functional change (unless sarah disagrees)
 1.57 20-May-2009  pooka Introduce -c [12], which can be used to open 1 or 2 ssh connections
to the server. If "2" is specified, a separate connection is used
for data and directory operations. Using two connections can
significantly increase directory operation performance on a saturated
link, at least up to 30x faster.
 1.56 29-Mar-2009  pooka Save errno before descending to cleanup gehennom.
 1.55 01-Jan-2009  pooka branches: 1.55.2;
Track parent directory in cross-directory renames so that mv & rm
for the same file during a vnode's lifecycle works.

... rename fix .. it's going to be a good year ...
 1.54 12-Aug-2008  pooka branches: 1.54.4;
void * -> puffs_cookie_t
no functional change
 1.53 09-Dec-2007  pooka branches: 1.53.8;
When creating a node, do the server-dance before creating the local
data structure. This avoids a race between create and lookup, as
the kernel now unlock the directory before the create operation
returns.
 1.52 07-Dec-2007  pooka Don't use puffs_cc_getusermount()
 1.51 07-Dec-2007  pooka Get rid of a lot of extra complexity due to doing separate getattr
requests in readdir - just use the results from the readdir rpc.
 1.50 07-Dec-2007  pooka In case a node is created in lookup in the readdir path, initialize
attribute cache timestamp to be equal to that of the directory
cache timestamp.
 1.49 04-Dec-2007  pooka When scheduling waiters for re-run, remove them from the waitqueue
already when scheduling to make sure they're not scheduled more
than once.
 1.48 30-Nov-2007  pooka Rototill.

Ok, ok, a few more words about it: stop holding puffs_cc as a holy
value and passing it around to almost every possible place (popquiz:
which kernel variable does this remind you of?). Instead, pass
the natural choice, puffs_usermount, and fetch puffs_cc via
puffs_cc_getcc() only in routines which actually need it. This
not only simplifies code, but (thanks to the introduction of
puffs_cc_getcc()) enables constructs which weren't previously sanely
possible, say layering as a curious example.

There's still a little to do on this front, but this was the major
fs interface blast.
 1.47 30-Nov-2007  pooka Introduce the concept of lazyopens. This means that when a file
handle open is requested, it is waited for only if the node was
not previously succesfully opened. The actual wait for the file
handle happens only when the file handle is actually needed (read
or write). This in turn has the effect that reading cached files
will be quick instead of waiting for the file handle from the sftp
server first. The wait previously could be very long if there were
serveral hundred k of outstanding requests in a limited-bandwidth
link.

The code is in some need of serious handholding, but it works, so
I'll leave that as "future work".
 1.46 27-Nov-2007  pooka Remove "puffs_cid" from the puffs interface following l-removal
from the kernel vfs interfaces. puffs_cc_getcaller(pcc) can be
used now should the same information be desired.
 1.45 18-Nov-2007  pooka Don't bother tracking opencount and try to close file handles in
the close method - simply just wait for inactive and do it there.
 1.44 11-Nov-2007  pooka Don't overload "attributes last read" for the symlink cache,
use a separate variable.
 1.43 10-Nov-2007  pooka Cache readlink results similarly to attributes. Makes "cd" fast
for a slow connection if a symlink is along the path.

(this should be done elsewhere, but I want it for sshfs now)
 1.42 21-Oct-2007  pooka Use puffs_cc_schedule() (which is an exported interface) instead of
plugging directly to puffs_goto().
 1.41 20-Oct-2007  pooka Add option -r to specify maximum number of active read requests
for each node. Setting this to a small number can be used to
improve interactive performance on low-bandwidth links when performing
bulk data reads. Of course I could also open separate pipes for
bulk and other, but this was quicker and less intrusive and doesn't
require authenticating twice.
 1.40 16-Oct-2007  pooka Close file handles if opencount drops to zero instead of closing
them if it does not drop to zero, i.e. fix a pretty obvious bug.
 1.39 24-Aug-2007  pooka branches: 1.39.2;
If a node is forcibly reclaimed, close can be called more times
than open (or less, for the particular fs). So don't be uptight
about opencount.
 1.38 23-Aug-2007  pooka Prevent crash in case there is more than one reader for the same
directory concurrently. Might need to revisit the method used here
later, though.
 1.37 29-Jul-2007  pooka branches: 1.37.4;
make closehandles() void, because that's what it is
 1.36 27-Jul-2007  pooka Track memory mappings. Close file handles already in close() if there
are no active mappings, otherwise do the standard inactive() run.
 1.35 19-Jul-2007  pooka Oops, the cookies for nfs of course represent the *next* directory
entry offset, not the current one.
 1.34 16-Jul-2007  pooka If the target node exists in rename, do not try to issue a setback
to signal no references, as that is not currently supported for
node_rename(). The removed node will not immediately be reclaimed,
but we can live with that for now.

While here, factor the removal code a bit to share with remove and
rmdir.

fixes PR kern/36637 by reinoud
 1.33 02-Jul-2007  pooka adapt: inactive no longer comes with int *refcount (not that it was
used except for setting the default here anyway)
 1.32 01-Jul-2007  pooka adapt: **newnode etc. pointers -> struct puffs_newinfo
 1.31 01-Jul-2007  pooka adapt: pid -> const struct puffs_cid *
 1.30 06-Jun-2007  pooka Move puffs to a two clause license where it already isn't so. And
as agc pointed out, even files with the third clause were already
effectively two clause because of a slight bug in the language...
 1.29 18-May-2007  pooka remove obvious incorrect assert() from doreclaim() and remove the
whole PSN_NUKED bit - it's not very useful because of the noref setback.
 1.28 18-May-2007  pooka * use NOREF setback to prompt immediate (well, once the kernel
reference count goes to 0) reclaim of deleted nodes as opposed to
waiting for the system to start reclaiming the freelist
* combine some nodeflags from different variable to one status variable
 1.27 15-May-2007  pooka adapt to "step 3" of puffs_framebuf changes
 1.26 11-May-2007  pooka adapt to puffs_framebuf changes
 1.25 07-May-2007  pooka Use inactive-on-demand, set inactive request in open() to close
file handles when last reference is removed from the kernel.
 1.24 06-May-2007  pooka Open files in open and close them in inactive. This avoids the
silly and problematic chmod + open + "un"chmod in write, among
other things.
 1.23 06-May-2007  pooka Fire off FXP_CLOSE in create() and return immediately instead of
waiting for the result.
 1.22 05-May-2007  pooka Adapt to use puffs_framebuf. This gets rid of most of the duplicated
code between mount_psshfs and mount_9p and clarifies the code
structure.
 1.21 01-May-2007  pooka g/c forgotten printf
 1.20 19-Apr-2007  pooka use puffs_cc_getspecific() where puffs_usermount is not required
 1.19 17-Apr-2007  pooka use PUFFS_STORE_DCOOKIE
 1.18 12-Apr-2007  pooka Actually, we can't reclaim our memory image for nodes which have
been nodetofh translated even if they are not valid on the sftp
server anymore, because some nfs client might still be clinging on
to the file handle we are reclaiming now.
 1.17 12-Apr-2007  pooka Support nfs exporting.

Now, when I say support, I mean "support", due to the limitations
of the backend. File handles are valid only for one session, since
nodes can only be identified by pathnames and pathnames don't (all)
fit into the nfs file handle space. Additionally, we can't detect
if a pathname is completely replaced by another file (if it's done
via some other route that through our mount, of course). But then
again, that's an inherent problem with sshfs even without nfs.
 1.16 12-Apr-2007  pooka Hide struct puffs_usermount from the rest of the world and provide
accessors for interesting data in it. Namely, you can now get
pu->pu_privdata with puffs_getspecific(), pu->pu_pn_root with
puffs_set/getroot() and pu->pu_maxreqlen with puffs_getmaxreqlen().
 1.15 11-Apr-2007  pooka adapt to new readdir signature. no functional change
 1.14 09-Apr-2007  pooka fix another case of checking for return value from wrong variable
(that's what you get when you copypaste code, a cid with a pin
to burst your bubble, that's what you get for all your troubles, I'll
never copypaste again)

CID 4461
 1.13 09-Apr-2007  pooka Check for success from correct variable after allocnode()

CID 4460
 1.12 22-Mar-2007  pooka If we have to do a stat in lookup, update time of last stat for
the new node to avoid a most likely immediate stat from the server.
 1.11 22-Mar-2007  pooka In case lookup from the directory listing fails, try statting the
target before giving up. Makes it possible to access files from
-r+x directories.
 1.10 13-Mar-2007  pooka * don't require a directory entry to exist for a reclaimee node.
this can happen legally when a file is removed from backing
storage not using this sshfs instance, a readdir is executed for
the parent directory and only then the node gets reclaimed.
* now that there is a mechanism in place which does not require a
pcc to do an sftp transaction, do not yield() in operations where
the final transaction is something where we don't care about the
return value (e.g. close handle). speedup benefit for no cost.
 1.9 27-Feb-2007  pooka size_t -> uint32_t cleanup to make LP64 build

noticed by wiz
 1.8 15-Feb-2007  pooka Support reclaim. To support dotdot lookup, here we leave the
reclaimed nodes hanging until all their children have been reclaimed
and then reclaim everything we can as far up to root as possible.
This is because the file system structures are currently interlinked
in a fashion which would make dotdot lookup based on purely a path
instead of a in-memory node parent member pointer very difficult.
Yes, this deserves a closer look some day.
 1.7 10-Feb-2007  pooka Don't destroy puffs_nodes in success paths of remove and rmdir: the
kernel assumes that they will not be recycled until reclaim.
 1.6 09-Feb-2007  pooka Implement "superreaddir". This issues a getattr for all the
directory entries already in readdir and caches the results instead
of waiting for each individial getattr from the kernel. For
high-latency links the difference in "ls -l" is quite astounding
and even on my lan "ls -lR" is faster than for nfs in a normal
directory hierarchy (i.e. not one artifically setup to have thousands
of files per directory).

TODO: implement some sort of bandwidth/latency measurement in the
code and enable or disable this option based on than information
(and a command-line flag).
 1.5 15-Jan-2007  pooka * adapt to libpuffs path changes
* fix bug in rename
 1.4 11-Jan-2007  pooka Give nukenode a destroy flag. If given, it completely wipes out
the node and directory cache (for directories). Otherwise it just
nukes directory entry information. The latter case is used when
renaming.
 1.3 07-Jan-2007  pooka * Adapt to openssh standard of timestamps in file attributes. Maybe
should make this work with the IETF standard some day, also.
* kludge with writes and permissions a bit to be able to flush data
cached in ubc to files which are already with r/o permissions in
the backend
 1.2 01-Jan-2007  pooka SUSv3 says: should the implementation not support directory unlinking,
unlink shall fail with EPERM and the errno of the failure shall be EPERM
 1.1 29-Dec-2006  pooka puffs sshfs, an sshfs implementation built on top of the puffs
continuation framework
 1.37.4.2 29-Jul-2007  pooka make closehandles() void, because that's what it is
 1.37.4.1 29-Jul-2007  pooka file node.c was added on branch matt-mips64 on 2007-07-29 17:50:23 +0000
 1.39.2.2 09-Jan-2008  matt sync with HEAD
 1.39.2.1 06-Nov-2007  matt sync with HEAD
 1.53.8.1 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.54.4.1 16-Jan-2009  snj Pull up following revision(s) (requested by pooka in ticket #254):
usr.sbin/puffs/mount_psshfs/node.c: revision 1.55
Track parent directory in cross-directory renames so that mv & rm
for the same file during a vnode's lifecycle works.
... rename fix .. it's going to be a good year ...
 1.55.2.1 13-May-2009  jym Sync with HEAD.

Third (and last) commit. See http://mail-index.netbsd.org/source-changes/2009/05/13/msg221222.html
 1.63.2.1 17-Apr-2012  yamt sync with head
 1.64.22.1 04-Nov-2016  pgoyette Sync with HEAD
 1.19 04-Nov-2012  christos include the headers you use
 1.18 08-Jan-2010  pooka branches: 1.18.6; 1.18.12;
Fix compilation with -Wsign-compare. Apparently 5.0 doesn't have
-Wsign-compare enabled, which explains a thing or two ...

pointed out by gson
 1.17 07-Jan-2010  pooka Fix chgrp (and don't rely on VNOVAL being -1, however unlikely that
is to be broken).
 1.16 07-Jan-2010  pooka Add -u and -g, which allow to remap one (local,remote) uidgid, i.e.
umapfs without the fuss.
 1.15 05-Nov-2009  pooka more lp64 it's-a-sign! fixes
 1.14 20-May-2009  pooka Copyright year management. no functional change (unless sarah disagrees)
 1.13 06-Sep-2008  pooka Due to a recent encounter with a sucky internet connection, add
experimental option -p, which tries to reestablish the connection
to the sftp server in case it is lost. This currently has a few
limitations (found in the man page), but generally works in some
use cases.

Better support might eventually emerge, but since that requires a
plunge into the depths of puffs_framebuf, I need quite a bit of
Fernet Branca to build up my courage before attempting it.
 1.12 06-Sep-2007  pooka branches: 1.12.8;
adapt to new cmpfb signature
 1.11 26-Jun-2007  pooka branches: 1.11.4;
Fix return value comparison inversion. Makes psshfs work for the
nocache case again, as the server error value for EOF is properly
handled again.
 1.10 06-Jun-2007  pooka Move puffs to a two clause license where it already isn't so. And
as agc pointed out, even files with the third clause were already
effectively two clause because of a slight bug in the language...
 1.9 20-May-2007  pooka Change the signature of puffs_respcmp to return -1,0,1 for smaller,
equal, larger, respectively instead of 0/1 for non/equal. This
will allow sorting the buffers for faster matching in libpuffs.
While here, change the name from respcmp to framecmp, as that better
reflects the purpose.

NOTE! there is no obvious way to make compilation fail for file
systems which may already be using this feature (although I don't
think there are any outside our tree, as the feature is two weeks
old). Nevertheless, non-updated file systems will fail very quickly.
 1.8 15-May-2007  pooka adapt to "step 3" of puffs_framebuf changes
 1.7 12-May-2007  pooka uint32_t vs. size_t pointer adjustments for LP64 build
 1.6 06-May-2007  pooka use puffs_framebuf_remaining()
 1.5 05-May-2007  pooka Adapt to use puffs_framebuf. This gets rid of most of the duplicated
code between mount_psshfs and mount_9p and clarifies the code
structure.
 1.4 18-Apr-2007  pooka Use ntohl instead of htonl when converting data from the network.
Result isn't that much different, though ....
 1.3 27-Feb-2007  pooka size_t -> uint32_t cleanup to make LP64 build

noticed by wiz
 1.2 07-Jan-2007  pooka * Adapt to openssh standard of timestamps in file attributes. Maybe
should make this work with the IETF standard some day, also.
* kludge with writes and permissions a bit to be able to flush data
cached in ubc to files which are already with r/o permissions in
the backend
 1.1 29-Dec-2006  pooka puffs sshfs, an sshfs implementation built on top of the puffs
continuation framework
 1.11.4.1 06-Nov-2007  matt sync with HEAD
 1.12.8.1 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.18.12.1 20-Nov-2012  tls Resync to 2012-11-19 00:00:00 UTC
 1.18.6.1 16-Jan-2013  yamt sync with (a bit old) head
 1.67 05-Dec-2021  msaitoh s/systme/system/ in comment.
 1.66 04-Nov-2012  christos include the headers you use
 1.65 31-Aug-2011  joerg branches: 1.65.2; 1.65.8;
Use __dead
 1.64 25-Aug-2011  jakllsch Bring mount_psshfs user/host/path argument processing in line with sftp(1).
 1.63 19-May-2011  riastradh Initialize psshfs root node's readdir waiters queue.

Every other node's readder waiters queue is initialized in makenode,
but the root node is created specially.

ok pooka
 1.62 29-Oct-2010  pooka Make error message more userfriendly in cases where server does
not support sftp.
 1.61 17-Feb-2010  pooka umgah. meant fts, not fsu
 1.60 17-Feb-2010  pooka Bump the initial rootdir link count guess to a ludicrous value.
 1.59 12-Jan-2010  pooka Make r/w servers unmount themselves cleanly upon receiving SIGINT/TERM.
 1.58 07-Jan-2010  pooka errx() already contains progname ...
 1.57 07-Jan-2010  pooka Don't allow mangling of uid or gid -1, since they have a special
meaning in the sftp protocol.
 1.56 07-Jan-2010  pooka Add -u and -g, which allow to remap one (local,remote) uidgid, i.e.
umapfs without the fuss.
 1.55 05-Nov-2009  pooka const woes
 1.54 20-May-2009  pooka wait() for dead ssh processes (but don't block, so we might have
one teeny zmobie in the system).
 1.53 20-May-2009  pooka Fix SNAFU in previous: connection handshake is already handled by
the connect routine now, so we're not required/allowed to do it twice.
Makes reconnect on dead link work again.
 1.52 20-May-2009  pooka Copyright year management. no functional change (unless sarah disagrees)
 1.51 20-May-2009  pooka Introduce -c [12], which can be used to open 1 or 2 ssh connections
to the server. If "2" is specified, a separate connection is used
for data and directory operations. Using two connections can
significantly increase directory operation performance on a saturated
link, at least up to 30x faster.
 1.50 23-Feb-2009  pooka Support statvfs@openssh.com extension where available. Makes df(1)
return something other than just 0.
caveat: statvfs is done for the mountpoint path, so might not give
the truth about a directory inside the mountpoint.
 1.49 06-Sep-2008  pooka branches: 1.49.6;
Due to a recent encounter with a sucky internet connection, add
experimental option -p, which tries to reestablish the connection
to the sftp server in case it is lost. This currently has a few
limitations (found in the man page), but generally works in some
use cases.

Better support might eventually emerge, but since that requires a
plunge into the depths of puffs_framebuf, I need quite a bit of
Fernet Branca to build up my courage before attempting it.
 1.48 11-Aug-2008  pooka call puffs_exit()
 1.47 14-Dec-2007  jmmv branches: 1.47.6;
Add a -F option to allow passing a configuration file to ssh(1), instead
of having to use multiple -O options on the command line. OKed by pooka@.
 1.46 30-Nov-2007  pooka Introduce the concept of lazyopens. This means that when a file
handle open is requested, it is waited for only if the node was
not previously succesfully opened. The actual wait for the file
handle happens only when the file handle is actually needed (read
or write). This in turn has the effect that reading cached files
will be quick instead of waiting for the file handle from the sftp
server first. The wait previously could be very long if there were
serveral hundred k of outstanding requests in a limited-bandwidth
link.

The code is in some need of serious handholding, but it works, so
I'll leave that as "future work".
 1.45 18-Nov-2007  pooka Don't bother tracking opencount and try to close file handles in
the close method - simply just wait for inactive and do it there.
 1.44 16-Nov-2007  pooka use puffs_daemon() instead of daemon()
 1.43 11-Nov-2007  pooka Don't overload "attributes last read" for the symlink cache,
use a separate variable.
 1.42 08-Nov-2007  pooka Make it possible to force attribute/directory cache re-read by
sending SIGHUP to mount_psshfs.
 1.41 08-Nov-2007  pooka don't allow negative timeouts apart from -1
 1.40 08-Nov-2007  pooka Make attribute & directory content caching timeout a command line
option. Use -t -1 to never expire, which is nifty on a high-latency
link where you know the server won't be modified from under you.
 1.39 06-Nov-2007  pooka Open the kernel descriptor as part of mount(), not init(). Then
it doesn't matter if someone fork()s or does other tricks between
init() and mount() (and besides, now it's where it logically should be).
 1.38 05-Nov-2007  pooka Actually, daemonize the file servers before mounting. I might require
at some point that the local protocol handler does not change after
the file system has been mounted.
 1.37 05-Nov-2007  pooka Pull the daemonizing code out of the library mainloop into the file
servers. Calling daemon() (i.e. fork()ing) inside a library can
cause nice surprises for e.g. threaded programs. As discussed with
Greg Oster & others.
 1.36 20-Oct-2007  pooka Add option -r to specify maximum number of active read requests
for each node. Setting this to a small number can be used to
improve interactive performance on low-bandwidth links when performing
bulk data reads. Of course I could also open separate pipes for
bulk and other, but this was quicker and less intrusive and doesn't
require authenticating twice.
 1.35 01-Sep-2007  pooka Adapt to new puffs_framev_init() signature - gotfb.
 1.34 27-Jul-2007  pooka branches: 1.34.4; 1.34.6;
Track memory mappings. Close file handles already in close() if there
are no active mappings, otherwise do the standard inactive() run.
 1.33 17-Jul-2007  pooka * add mntfromname parameter to puffs_init()
* set it in file servers
 1.32 07-Jul-2007  pooka I'm fairly certain this file system's type is psshfs, not ppshfs
(how did that manage to be there this long?)
 1.31 07-Jul-2007  pooka adapt to puffs_framebuf changes
 1.30 26-Jun-2007  pooka g/c bit about MNT_NOATIME being mandatory. it obvious isn't since
the code was #if 0'd out.
 1.29 06-Jun-2007  pooka Move puffs to a two clause license where it already isn't so. And
as agc pointed out, even files with the third clause were already
effectively two clause because of a slight bug in the language...
 1.28 21-May-2007  pooka devnullify ssh process's stderr to avoid tty spam
 1.27 17-May-2007  pooka reflect changes in fs mount: use puffs_mount() instead of
puffs_domount() & puffs_start()
 1.26 15-May-2007  pooka adapt to "step 3" of puffs_framebuf changes
 1.25 11-May-2007  pooka use puffs_mainloop(); puffs_framebuf_eventloop() is a goner
 1.24 11-May-2007  pooka adapt to puffs_framebuf changes
 1.23 09-May-2007  pooka use fprintf instead of errx for usage()
 1.22 09-May-2007  tnn Change [-O sshopt value] to [-O sshopt=value] to better conform with the
way other userland apps behave. In /etc/fstab one would specify options
with -O=sshopt=value.
 1.21 09-May-2007  tnn Add flag [-O sshopt value] to mount_psshfs(8), allowing the user to specify
options to pass along to ssh(1). Remove redundant flag [-p sshport]; an
equivalent of -O Port sshport. ok'd by pooka@
 1.20 07-May-2007  pooka Use inactive-on-demand, set inactive request in open() to close
file handles when last reference is removed from the kernel.
 1.19 06-May-2007  pooka Open files in open and close them in inactive. This avoids the
silly and problematic chmod + open + "un"chmod in write, among
other things.
 1.18 06-May-2007  pooka adapt to puffs_framebuf_eventloop signature change
 1.17 05-May-2007  pooka Adapt to use puffs_framebuf. This gets rid of most of the duplicated
code between mount_psshfs and mount_9p and clarifies the code
structure.
 1.16 02-May-2007  pooka For consistency destroy kernel request putbuffers also when
escaping event loop (the process is going to exit anyway, but ...)
 1.15 29-Apr-2007  pooka Add -p to specify server port if someone should for some reason
happen to run sshd on port, to pick a random number, 443.

Probably should give a way to specify rest of the ssh options.
 1.14 22-Apr-2007  pooka Use PUFFS_KFLAG_WTCACHE to avoid the need to do a sync(8) on the
client host to get all the data onto the server.
 1.13 16-Apr-2007  pooka adapt to kernel changes with file handles
 1.12 13-Apr-2007  pooka Instead of keeping on adding parameters to puffs_mount(), make it
only take the bare essentials, which currently means removing
"maxreqlen" from the argument list (all current callers I'm aware
of set it as 0 anyway). Introduce puffs_init(), which provides a
context for setting various parameters and puffs_domount(), which
can be used to mount the file system. Keep puffs_mount() as a
shortcut for the above two for simple file systems.

Bump development ABI version to 13. After all, it's Friday the 13th.
Watch out! Bad things can happen on Friday the 13th. --No carrier--
 1.11 12-Apr-2007  pooka Support nfs exporting.

Now, when I say support, I mean "support", due to the limitations
of the backend. File handles are valid only for one session, since
nodes can only be identified by pathnames and pathnames don't (all)
fit into the nfs file handle space. Additionally, we can't detect
if a pathname is completely replaced by another file (if it's done
via some other route that through our mount, of course). But then
again, that's an inherent problem with sshfs even without nfs.
 1.10 12-Apr-2007  pooka Hide struct puffs_usermount from the rest of the world and provide
accessors for interesting data in it. Namely, you can now get
pu->pu_privdata with puffs_getspecific(), pu->pu_pn_root with
puffs_set/getroot() and pu->pu_maxreqlen with puffs_getmaxreqlen().
 1.9 13-Mar-2007  pooka * don't require a directory entry to exist for a reclaimee node.
this can happen legally when a file is removed from backing
storage not using this sshfs instance, a readdir is executed for
the parent directory and only then the node gets reclaimed.
* now that there is a mechanism in place which does not require a
pcc to do an sftp transaction, do not yield() in operations where
the final transaction is something where we don't care about the
return value (e.g. close handle). speedup benefit for no cost.
 1.8 15-Feb-2007  pooka Support reclaim. To support dotdot lookup, here we leave the
reclaimed nodes hanging until all their children have been reclaimed
and then reclaim everything we can as far up to root as possible.
This is because the file system structures are currently interlinked
in a fashion which would make dotdot lookup based on purely a path
instead of a in-memory node parent member pointer very difficult.
Yes, this deserves a closer look some day.
 1.7 09-Feb-2007  pooka Implement "superreaddir". This issues a getattr for all the
directory entries already in readdir and caches the results instead
of waiting for each individial getattr from the kernel. For
high-latency links the difference in "ls -l" is quite astounding
and even on my lan "ls -lR" is faster than for nfs in a normal
directory hierarchy (i.e. not one artifically setup to have thousands
of files per directory).

TODO: implement some sort of bandwidth/latency measurement in the
code and enable or disable this option based on than information
(and a command-line flag).
 1.6 20-Jan-2007  pooka more ketchup
 1.5 20-Jan-2007  pooka play catchup with the lib
 1.4 11-Jan-2007  pooka don't detach if -s or -o dump is given on the command line
 1.3 07-Jan-2007  pooka use UBC by default
 1.2 06-Jan-2007  pooka * do mount as a forward operation instead of a callback
* process -o args for mount
 1.1 29-Dec-2006  pooka puffs sshfs, an sshfs implementation built on top of the puffs
continuation framework
 1.34.6.2 27-Jul-2007  pooka Track memory mappings. Close file handles already in close() if there
are no active mappings, otherwise do the standard inactive() run.
 1.34.6.1 27-Jul-2007  pooka file psshfs.c was added on branch matt-mips64 on 2007-07-27 09:46:28 +0000
 1.34.4.2 09-Jan-2008  matt sync with HEAD
 1.34.4.1 06-Nov-2007  matt sync with HEAD
 1.47.6.1 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.49.6.1 13-May-2009  jym Sync with HEAD.

Third (and last) commit. See http://mail-index.netbsd.org/source-changes/2009/05/13/msg221222.html
 1.65.8.1 20-Nov-2012  tls Resync to 2012-11-19 00:00:00 UTC
 1.65.2.1 16-Jan-2013  yamt sync with (a bit old) head
 1.40 01-Apr-2010  pooka In case we create a file, reclaim the vnode, and lookup the file
without the directory timeout expiring, we get vattr_null as the
attributes for that file. Ensure that we always report sane
attributes by getting them from the server if this is the case.
(also, sprinkle some const)

Fixes problem reported by dyoung. But wait! The bug's medallion
begins to glow! The bug looks much better! I crumble to dust.

There's probably another similar bug related to "lazy open". It
will trigger if we reclaim a node before the response to the open
arrives. Even the comments (typoless) know about this bug ...
But verifying it exists and fixing it will have to wait for another
day.
 1.39 07-Jan-2010  pooka Add -u and -g, which allow to remap one (local,remote) uidgid, i.e.
umapfs without the fuss.
 1.38 05-Nov-2009  pooka more lp64 it's-a-sign! fixes
 1.37 20-May-2009  pooka Copyright year management. no functional change (unless sarah disagrees)
 1.36 20-May-2009  pooka Introduce -c [12], which can be used to open 1 or 2 ssh connections
to the server. If "2" is specified, a separate connection is used
for data and directory operations. Using two connections can
significantly increase directory operation performance on a saturated
link, at least up to 30x faster.
 1.35 23-Feb-2009  pooka Support statvfs@openssh.com extension where available. Makes df(1)
return something other than just 0.
caveat: statvfs is done for the mountpoint path, so might not give
the truth about a directory inside the mountpoint.
 1.34 06-Sep-2008  pooka branches: 1.34.6;
Due to a recent encounter with a sucky internet connection, add
experimental option -p, which tries to reestablish the connection
to the sftp server in case it is lost. This currently has a few
limitations (found in the man page), but generally works in some
use cases.

Better support might eventually emerge, but since that requires a
plunge into the depths of puffs_framebuf, I need quite a bit of
Fernet Branca to build up my courage before attempting it.
 1.33 07-Dec-2007  pooka branches: 1.33.8;
Don't use puffs_cc_getusermount()
 1.32 07-Dec-2007  pooka Get rid of a lot of extra complexity due to doing separate getattr
requests in readdir - just use the results from the readdir rpc.
 1.31 30-Nov-2007  pooka Rototill.

Ok, ok, a few more words about it: stop holding puffs_cc as a holy
value and passing it around to almost every possible place (popquiz:
which kernel variable does this remind you of?). Instead, pass
the natural choice, puffs_usermount, and fetch puffs_cc via
puffs_cc_getcc() only in routines which actually need it. This
not only simplifies code, but (thanks to the introduction of
puffs_cc_getcc()) enables constructs which weren't previously sanely
possible, say layering as a curious example.

There's still a little to do on this front, but this was the major
fs interface blast.
 1.30 30-Nov-2007  pooka Introduce the concept of lazyopens. This means that when a file
handle open is requested, it is waited for only if the node was
not previously succesfully opened. The actual wait for the file
handle happens only when the file handle is actually needed (read
or write). This in turn has the effect that reading cached files
will be quick instead of waiting for the file handle from the sftp
server first. The wait previously could be very long if there were
serveral hundred k of outstanding requests in a limited-bandwidth
link.

The code is in some need of serious handholding, but it works, so
I'll leave that as "future work".
 1.29 18-Nov-2007  pooka Don't bother tracking opencount and try to close file handles in
the close method - simply just wait for inactive and do it there.
 1.28 11-Nov-2007  pooka Don't overload "attributes last read" for the symlink cache,
use a separate variable.
 1.27 10-Nov-2007  pooka Cache readlink results similarly to attributes. Makes "cd" fast
for a slow connection if a symlink is along the path.

(this should be done elsewhere, but I want it for sshfs now)
 1.26 08-Nov-2007  pooka Make attribute & directory content caching timeout a command line
option. Use -t -1 to never expire, which is nifty on a high-latency
link where you know the server won't be modified from under you.
 1.25 20-Oct-2007  pooka Add option -r to specify maximum number of active read requests
for each node. Setting this to a small number can be used to
improve interactive performance on low-bandwidth links when performing
bulk data reads. Of course I could also open separate pipes for
bulk and other, but this was quicker and less intrusive and doesn't
require authenticating twice.
 1.24 06-Sep-2007  pooka adapt to new cmpfb signature
 1.23 23-Aug-2007  pooka branches: 1.23.2;
Prevent crash in case there is more than one reader for the same
directory concurrently. Might need to revisit the method used here
later, though.
 1.22 27-Jul-2007  pooka branches: 1.22.4;
Track memory mappings. Close file handles already in close() if there
are no active mappings, otherwise do the standard inactive() run.
 1.21 07-Jul-2007  pooka adapt to puffs_framebuf changes
 1.20 06-Jun-2007  pooka Move puffs to a two clause license where it already isn't so. And
as agc pointed out, even files with the third clause were already
effectively two clause because of a slight bug in the language...
 1.19 20-May-2007  pooka Observe that when doing ls -l, the attributes of the first file
getattr are usually still outstanding when we already would like
the result. Instead of issueing another stat which will be serviced
only after all the other entries in the directory, record all the
outgoing readdir getattr buffers and if we encounter an outstanding
request when we need to fetch attrs, do a puffs_framev_framebuf_ccpromote()
wait for it instead of firing off the second query. This shaves
almost 10% off the time for ls -lR.

Also, get rid of the SUPERREADDIR conditional, since it has penetrated
the code quite a bit and the #ifdef SUPERREADDIRs were starting to
look like tagliatelle alla bolognese (n.b. I love how it looks,
but I wouldn't like it either if my tagliatelle alla bolognese
looked like psshfs code). Maybe it should be re-introduced in the
form of a switch?
 1.18 20-May-2007  pooka Implement SUPERREADDIR (like nfs readdirplus) getattr cache warming
a bit differently: when reading the directory, store all getattr
caching queries and fire off only when the directory read is
complete. That way the common sequence is not [readdir, lots of
async getattr requests, readdir EOF] but rather [readdir, readdir
EOF, lots of async getattr]. This speeds up ls -lR by about 25%
(on my LAN).
 1.17 18-May-2007  pooka remove obvious incorrect assert() from doreclaim() and remove the
whole PSN_NUKED bit - it's not very useful because of the noref setback.
 1.16 18-May-2007  pooka * use NOREF setback to prompt immediate (well, once the kernel
reference count goes to 0) reclaim of deleted nodes as opposed to
waiting for the system to start reclaiming the freelist
* combine some nodeflags from different variable to one status variable
 1.15 15-May-2007  pooka adapt to "step 3" of puffs_framebuf changes
 1.14 12-May-2007  pooka uint32_t vs. size_t pointer adjustments for LP64 build
 1.13 11-May-2007  pooka adapt to puffs_framebuf changes
 1.12 06-May-2007  pooka Open files in open and close them in inactive. This avoids the
silly and problematic chmod + open + "un"chmod in write, among
other things.
 1.11 05-May-2007  pooka Adapt to use puffs_framebuf. This gets rid of most of the duplicated
code between mount_psshfs and mount_9p and clarifies the code
structure.
 1.10 19-Apr-2007  pooka use puffs_cc_getspecific() where puffs_usermount is not required
 1.9 12-Apr-2007  pooka Support nfs exporting.

Now, when I say support, I mean "support", due to the limitations
of the backend. File handles are valid only for one session, since
nodes can only be identified by pathnames and pathnames don't (all)
fit into the nfs file handle space. Additionally, we can't detect
if a pathname is completely replaced by another file (if it's done
via some other route that through our mount, of course). But then
again, that's an inherent problem with sshfs even without nfs.
 1.8 12-Apr-2007  pooka Hide struct puffs_usermount from the rest of the world and provide
accessors for interesting data in it. Namely, you can now get
pu->pu_privdata with puffs_getspecific(), pu->pu_pn_root with
puffs_set/getroot() and pu->pu_maxreqlen with puffs_getmaxreqlen().
 1.7 22-Mar-2007  pooka In case lookup from the directory listing fails, try statting the
target before giving up. Makes it possible to access files from
-r+x directories.
 1.6 27-Feb-2007  pooka size_t -> uint32_t cleanup to make LP64 build

noticed by wiz
 1.5 15-Feb-2007  pooka Support reclaim. To support dotdot lookup, here we leave the
reclaimed nodes hanging until all their children have been reclaimed
and then reclaim everything we can as far up to root as possible.
This is because the file system structures are currently interlinked
in a fashion which would make dotdot lookup based on purely a path
instead of a in-memory node parent member pointer very difficult.
Yes, this deserves a closer look some day.
 1.4 09-Feb-2007  pooka Implement "superreaddir". This issues a getattr for all the
directory entries already in readdir and caches the results instead
of waiting for each individial getattr from the kernel. For
high-latency links the difference in "ls -l" is quite astounding
and even on my lan "ls -lR" is faster than for nfs in a normal
directory hierarchy (i.e. not one artifically setup to have thousands
of files per directory).

TODO: implement some sort of bandwidth/latency measurement in the
code and enable or disable this option based on than information
(and a command-line flag).
 1.3 15-Jan-2007  pooka * adapt to libpuffs path changes
* fix bug in rename
 1.2 06-Jan-2007  pooka * do mount as a forward operation instead of a callback
* process -o args for mount
 1.1 29-Dec-2006  pooka puffs sshfs, an sshfs implementation built on top of the puffs
continuation framework
 1.22.4.2 27-Jul-2007  pooka Track memory mappings. Close file handles already in close() if there
are no active mappings, otherwise do the standard inactive() run.
 1.22.4.1 27-Jul-2007  pooka file psshfs.h was added on branch matt-mips64 on 2007-07-27 09:46:29 +0000
 1.23.2.2 09-Jan-2008  matt sync with HEAD
 1.23.2.1 06-Nov-2007  matt sync with HEAD
 1.33.8.1 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.34.6.1 13-May-2009  jym Sync with HEAD.

Third (and last) commit. See http://mail-index.netbsd.org/source-changes/2009/05/13/msg221222.html
 1.2 06-Jun-2007  pooka Move puffs to a two clause license where it already isn't so. And
as agc pointed out, even files with the third clause were already
effectively two clause because of a slight bug in the language...
 1.1 29-Dec-2006  pooka puffs sshfs, an sshfs implementation built on top of the puffs
continuation framework
 1.51 04-Nov-2012  christos include the headers you use
 1.50 01-Apr-2010  pooka branches: 1.50.6; 1.50.12;
In case we create a file, reclaim the vnode, and lookup the file
without the directory timeout expiring, we get vattr_null as the
attributes for that file. Ensure that we always report sane
attributes by getting them from the server if this is the case.
(also, sprinkle some const)

Fixes problem reported by dyoung. But wait! The bug's medallion
begins to glow! The bug looks much better! I crumble to dust.

There's probably another similar bug related to "lazy open". It
will trigger if we reclaim a node before the response to the open
arrives. Even the comments (typoless) know about this bug ...
But verifying it exists and fixing it will have to wait for another
day.
 1.49 17-Feb-2010  pooka Save "." attributes in readdir. This fixes the root link count
after readdir is called for the root dir the first time (yes,
there's still a window of wrong link count after the fs is mounted.
it's currently quite difficult to call sftp_readdir() from outside
the main loop).

Should fix "find /mnt" problem for a mountpoint with more than the
"guessed" amount of subdirectories, as reported by dyoung.
 1.48 07-Jan-2010  pooka Add -u and -g, which allow to remap one (local,remote) uidgid, i.e.
umapfs without the fuss.
 1.47 05-Nov-2009  pooka more lp64 it's-a-sign! fixes
 1.46 20-May-2009  pooka Introduce -c [12], which can be used to open 1 or 2 ssh connections
to the server. If "2" is specified, a separate connection is used
for data and directory operations. Using two connections can
significantly increase directory operation performance on a saturated
link, at least up to 30x faster.
 1.45 13-Dec-2007  pooka Record inode number in psshfs_dir also for already instantiated
nodes when doing readdir. This makes pwd work again for cases
where getcwd() actually has to do the "READDIR + compare inode
numbers" trick.

Yet another problem reported by jmmv.
 1.44 13-Dec-2007  pooka *blink*, remember to update comment too
 1.43 13-Dec-2007  pooka Do previous in a way which causes less waste of potential inode numbers.
 1.42 12-Dec-2007  pooka Fix lossage noticed by jmmv (mostly for -t 0 mounts) and make sure
a node always has the inode number set. And since I'm feeling
generous, sprinkle a few comments around the affected areas (mostly
so that I'd remember what in the world the code is trying to do).
 1.41 07-Dec-2007  pooka Don't use puffs_cc_getusermount()
 1.40 07-Dec-2007  pooka Get rid of a lot of extra complexity due to doing separate getattr
requests in readdir - just use the results from the readdir rpc.
 1.39 05-Dec-2007  pooka Invalidate namecache for a directory only if we've already read it
at least once.
 1.38 30-Nov-2007  pooka Rototill.

Ok, ok, a few more words about it: stop holding puffs_cc as a holy
value and passing it around to almost every possible place (popquiz:
which kernel variable does this remind you of?). Instead, pass
the natural choice, puffs_usermount, and fetch puffs_cc via
puffs_cc_getcc() only in routines which actually need it. This
not only simplifies code, but (thanks to the introduction of
puffs_cc_getcc()) enables constructs which weren't previously sanely
possible, say layering as a curious example.

There's still a little to do on this front, but this was the major
fs interface blast.
 1.37 30-Nov-2007  pooka Introduce the concept of lazyopens. This means that when a file
handle open is requested, it is waited for only if the node was
not previously succesfully opened. The actual wait for the file
handle happens only when the file handle is actually needed (read
or write). This in turn has the effect that reading cached files
will be quick instead of waiting for the file handle from the sftp
server first. The wait previously could be very long if there were
serveral hundred k of outstanding requests in a limited-bandwidth
link.

The code is in some need of serious handholding, but it works, so
I'll leave that as "future work".
 1.36 16-Nov-2007  pooka If a regular file was remotely updated and a readdir was done on
the directory which contained the file before a getattr on the file
itself, the locally cached mtime would be updated without invalidating
the kernel page cache. Thus incorrect data would be returned when
the node was read afterwards as the node size wouldn't match the
data length in the page cache.

Fix the problem by making all vattr-setting routines use the same code.

Problem noticed again by jmmv & atf (and again by running atf over
psshfs ... sometimes you're the windshield, sometimes you're the bug)
 1.35 16-Nov-2007  jmmv Drop trailing whitespace. On behalf of pooka.
 1.34 16-Nov-2007  pooka When checking if the file page cache should be invalidated, compare
against cached mtime instead of attrread - attrread can be reset
these days by sending SIGHUP.

Problem noticed by jmmv & atf (well.. namely by using atf through psshfs).
 1.33 10-Nov-2007  pooka Cache readlink results similarly to attributes. Makes "cd" fast
for a slow connection if a symlink is along the path.

(this should be done elsewhere, but I want it for sshfs now)
 1.32 08-Nov-2007  pooka Make it possible to force attribute/directory cache re-read by
sending SIGHUP to mount_psshfs.
 1.31 08-Nov-2007  pooka Make attribute & directory content caching timeout a command line
option. Use -t -1 to never expire, which is nifty on a high-latency
link where you know the server won't be modified from under you.
 1.30 20-Oct-2007  pooka Add option -r to specify maximum number of active read requests
for each node. Setting this to a small number can be used to
improve interactive performance on low-bandwidth links when performing
bulk data reads. Of course I could also open separate pipes for
bulk and other, but this was quicker and less intrusive and doesn't
require authenticating twice.
 1.29 08-Sep-2007  pooka Free longname in all error branches. CID: 4545
 1.28 25-Aug-2007  pooka branches: 1.28.2;
error branch management. CID 4503
 1.27 25-Aug-2007  pooka jump to correct error branch. CID 4500
 1.26 23-Aug-2007  pooka Prevent crash in case there is more than one reader for the same
directory concurrently. Might need to revisit the method used here
later, though.
 1.25 21-Jun-2007  pooka print warning if directory cache invalidation fails
 1.24 06-Jun-2007  pooka Move puffs to a two clause license where it already isn't so. And
as agc pointed out, even files with the third clause were already
effectively two clause because of a slight bug in the language...
 1.23 20-May-2007  pooka In case we are really tight on vnodes, the directory vnode might
be reclaimed from under while we are warming the getattr cache.
Shuffle some code to prevent the effects. Theoretically the race
is still possible, but I don't think it will happen in practice.
In any case, the code could benefit from some more dusting.
 1.22 20-May-2007  pooka Observe that when doing ls -l, the attributes of the first file
getattr are usually still outstanding when we already would like
the result. Instead of issueing another stat which will be serviced
only after all the other entries in the directory, record all the
outgoing readdir getattr buffers and if we encounter an outstanding
request when we need to fetch attrs, do a puffs_framev_framebuf_ccpromote()
wait for it instead of firing off the second query. This shaves
almost 10% off the time for ls -lR.

Also, get rid of the SUPERREADDIR conditional, since it has penetrated
the code quite a bit and the #ifdef SUPERREADDIRs were starting to
look like tagliatelle alla bolognese (n.b. I love how it looks,
but I wouldn't like it either if my tagliatelle alla bolognese
looked like psshfs code). Maybe it should be re-introduced in the
form of a switch?
 1.21 20-May-2007  pooka Implement SUPERREADDIR (like nfs readdirplus) getattr cache warming
a bit differently: when reading the directory, store all getattr
caching queries and fire off only when the directory read is
complete. That way the common sequence is not [readdir, lots of
async getattr requests, readdir EOF] but rather [readdir, readdir
EOF, lots of async getattr]. This speeds up ls -lR by about 25%
(on my LAN).
 1.20 18-May-2007  pooka remove obvious incorrect assert() from doreclaim() and remove the
whole PSN_NUKED bit - it's not very useful because of the noref setback.
 1.19 18-May-2007  pooka * use NOREF setback to prompt immediate (well, once the kernel
reference count goes to 0) reclaim of deleted nodes as opposed to
waiting for the system to start reclaiming the freelist
* combine some nodeflags from different variable to one status variable
 1.18 16-May-2007  pooka Add error parameter to puffs_framev_cb to signal if there is a
result of if the function should just do resource cleanup.
 1.17 15-May-2007  pooka remember to free longname if getting vattr from the message fails

CID 4486
 1.16 15-May-2007  pooka adapt to "step 3" of puffs_framebuf changes
 1.15 11-May-2007  pooka adapt to puffs_framebuf changes
 1.14 05-May-2007  pooka Adapt to use puffs_framebuf. This gets rid of most of the duplicated
code between mount_psshfs and mount_9p and clarifies the code
structure.
 1.13 12-Apr-2007  pooka Hide struct puffs_usermount from the rest of the world and provide
accessors for interesting data in it. Namely, you can now get
pu->pu_privdata with puffs_getspecific(), pu->pu_pn_root with
puffs_set/getroot() and pu->pu_maxreqlen with puffs_getmaxreqlen().
 1.12 22-Mar-2007  pooka In case lookup from the directory listing fails, try statting the
target before giving up. Makes it possible to access files from
-r+x directories.
 1.11 13-Mar-2007  pooka * don't require a directory entry to exist for a reclaimee node.
this can happen legally when a file is removed from backing
storage not using this sshfs instance, a readdir is executed for
the parent directory and only then the node gets reclaimed.
* now that there is a mechanism in place which does not require a
pcc to do an sftp transaction, do not yield() in operations where
the final transaction is something where we don't care about the
return value (e.g. close handle). speedup benefit for no cost.
 1.10 27-Feb-2007  pooka one more size_t -> uint32_t
 1.9 15-Feb-2007  pooka Support reclaim. To support dotdot lookup, here we leave the
reclaimed nodes hanging until all their children have been reclaimed
and then reclaim everything we can as far up to root as possible.
This is because the file system structures are currently interlinked
in a fashion which would make dotdot lookup based on purely a path
instead of a in-memory node parent member pointer very difficult.
Yes, this deserves a closer look some day.
 1.8 09-Feb-2007  pooka Implement "superreaddir". This issues a getattr for all the
directory entries already in readdir and caches the results instead
of waiting for each individial getattr from the kernel. For
high-latency links the difference in "ls -l" is quite astounding
and even on my lan "ls -lR" is faster than for nfs in a normal
directory hierarchy (i.e. not one artifically setup to have thousands
of files per directory).

TODO: implement some sort of bandwidth/latency measurement in the
code and enable or disable this option based on than information
(and a command-line flag).
 1.7 15-Jan-2007  pooka * adapt to libpuffs path changes
* fix bug in rename
 1.6 11-Jan-2007  pooka Give nukenode a destroy flag. If given, it completely wipes out
the node and directory cache (for directories). Otherwise it just
nukes directory entry information. The latter case is used when
renaming.
 1.5 09-Jan-2007  pooka Invalidate kernel name cache for a directory when (re-)reading
directory contents from the server.
 1.4 09-Jan-2007  pooka Mark invalid attributes as invalid. Fixes stuff such a cp'ing files
when cp uses mmap(). Also makes attributes appear correct.
 1.3 07-Jan-2007  pooka properly carry fileid for directory entries, fixes pwd
 1.2 07-Jan-2007  pooka fix link count calculation for directories
 1.1 29-Dec-2006  pooka puffs sshfs, an sshfs implementation built on top of the puffs
continuation framework
 1.28.2.2 09-Jan-2008  matt sync with HEAD
 1.28.2.1 06-Nov-2007  matt sync with HEAD
 1.50.12.1 20-Nov-2012  tls Resync to 2012-11-19 00:00:00 UTC
 1.50.6.1 16-Jan-2013  yamt sync with (a bit old) head

RSS XML Feed