Enable WARNS=4 by default, except for:
cpuctl dumplfs hprop ipf iprop-log kadmin kcm kdc kdigest
kimpersonate kstash ktutil makefs ndbootd ntp pppd quot
racoon racoonctl rtadvd sntp sup tcpdchk tcpdmatch tcpdump
traceroute traceroute6 user veriexecgen wsmoused zic
(Mostly third-party applications)

branches: 1.3.16; 1.3.20;
get rid of DBG=

man page for mount_psshfs

puffs sshfs, an sshfs implementation built on top of the puffs
continuation framework

Restore binary compatibility by using the statvfs90 structure internally.

branches: 1.24.42;
remove some always true if () checks GCC 4.5.3 picks up.

Make error message more userfriendly in cases where server does
not support sftp.

In case we create a file, reclaim the vnode, and lookup the file
without the directory timeout expiring, we get vattr_null as the
attributes for that file. Ensure that we always report sane
attributes by getting them from the server if this is the case.
(also, sprinkle some const)

Fixes problem reported by dyoung. But wait! The bug's medallion
begins to glow! The bug looks much better! I crumble to dust.

There's probably another similar bug related to "lazy open". It
will trigger if we reclaim a node before the response to the open
arrives. Even the comments (typoless) know about this bug ...
But verifying it exists and fixing it will have to wait for another
day.

statvfs:

Since bresvd is not available from the server, calculate:
bresvd = bfree - bavail.

Then df calculates:
bavail = bfree - bresvd;

And we now get a much more accurate report from df on how much one
can write to the fs.

Copyright year management. no functional change (unless sarah disagrees)

Introduce -c [12], which can be used to open 1 or 2 ssh connections
to the server. If "2" is specified, a separate connection is used
for data and directory operations. Using two connections can
significantly increase directory operation performance on a saturated
link, at least up to 30x faster.

Support statvfs@openssh.com extension where available. Makes df(1)
return something other than just 0.
caveat: statvfs is done for the mountpoint path, so might not give
the truth about a directory inside the mountpoint.

branches: 1.17.6;
Due to a recent encounter with a sucky internet connection, add
experimental option -p, which tries to reestablish the connection
to the sftp server in case it is lost. This currently has a few
limitations (found in the man page), but generally works in some
use cases.

Better support might eventually emerge, but since that requires a
plunge into the depths of puffs_framebuf, I need quite a bit of
Fernet Branca to build up my courage before attempting it.

void * -> puffs_cookie_t
no functional change

branches: 1.15.8;
Rototill.

Ok, ok, a few more words about it: stop holding puffs_cc as a holy
value and passing it around to almost every possible place (popquiz:
which kernel variable does this remind you of?). Instead, pass
the natural choice, puffs_usermount, and fetch puffs_cc via
puffs_cc_getcc() only in routines which actually need it. This
not only simplifies code, but (thanks to the introduction of
puffs_cc_getcc()) enables constructs which weren't previously sanely
possible, say layering as a curious example.

There's still a little to do on this front, but this was the major
fs interface blast.

Remove "puffs_cid" from the puffs interface following l-removal
from the kernel vfs interfaces. puffs_cc_getcaller(pcc) can be
used now should the same information be desired.

branches: 1.13.4;
adapt: **newnode etc. pointers -> struct puffs_newinfo

adapt: pid -> const struct puffs_cid *

Move puffs to a two clause license where it already isn't so. And
as agc pointed out, even files with the third clause were already
effectively two clause because of a slight bug in the language...

* use NOREF setback to prompt immediate (well, once the kernel
reference count goes to 0) reclaim of deleted nodes as opposed to
waiting for the system to start reclaiming the freelist
* combine some nodeflags from different variable to one status variable

reflect changes in fs mount: use puffs_mount() instead of
puffs_domount() & puffs_start()

Adapt to use puffs_framebuf. This gets rid of most of the duplicated
code between mount_psshfs and mount_9p and clarifies the code
structure.

psbuf_recycle now exists, so use it instead of destroy&make in one
spot I probably missed originally. No real functional change.

Actually, we can't reclaim our memory image for nodes which have
been nodetofh translated even if they are not valid on the sftp
server anymore, because some nfs client might still be clinging on
to the file handle we are reclaiming now.

Support nfs exporting.

Now, when I say support, I mean "support", due to the limitations
of the backend. File handles are valid only for one session, since
nodes can only be identified by pathnames and pathnames don't (all)
fit into the nfs file handle space. Additionally, we can't detect
if a pathname is completely replaced by another file (if it's done
via some other route that through our mount, of course). But then
again, that's an inherent problem with sshfs even without nfs.

Hide struct puffs_usermount from the rest of the world and provide
accessors for interesting data in it. Namely, you can now get
pu->pu_privdata with puffs_getspecific(), pu->pu_pn_root with
puffs_set/getroot() and pu->pu_maxreqlen with puffs_getmaxreqlen().

* adapt to libpuffs path changes
* fix bug in rename

* do mount as a forward operation instead of a callback
* process -o args for mount

puffs sshfs, an sshfs implementation built on top of the puffs
continuation framework

s/stavfs/statvfs/

branches: 1.26.6;
fix spelling typo

branches: 1.25.2; 1.25.8;
Wording, from Snader_LB.

Give credit: psshfs was inspired by FUSE sshfs.

Don't be terse.

Add -u and -g, which allow to remap one (local,remote) uidgid, i.e.
umapfs without the fuss.

Copyright year management. no functional change (unless sarah disagrees)

Introduce -c [12], which can be used to open 1 or 2 ssh connections
to the server. If "2" is specified, a separate connection is used
for data and directory operations. Using two connections can
significantly increase directory operation performance on a saturated
link, at least up to 30x faster.

Make "sftp" a link.

add caveat about info displayed by df(1)

branches: 1.17.6;
Due to a recent encounter with a sucky internet connection, add
experimental option -p, which tries to reestablish the connection
to the sftp server in case it is lost. This currently has a few
limitations (found in the man page), but generally works in some
use cases.

Better support might eventually emerge, but since that requires a
plunge into the depths of puffs_framebuf, I need quite a bit of
Fernet Branca to build up my courage before attempting it.

branches: 1.16.6;
Add a -F option to allow passing a configuration file to ssh(1), instead
of having to use multiple -O options on the command line. OKed by pooka@.

Do not overload Xr macro.

Sort SEE ALSO. Minor nit fixes.

Make it possible to force attribute/directory cache re-read by
sending SIGHUP to mount_psshfs.

Make attribute & directory content caching timeout a command line
option. Use -t -1 to never expire, which is nifty on a high-latency
link where you know the server won't be modified from under you.

Add option -r to specify maximum number of active read requests
for each node. Setting this to a small number can be used to
improve interactive performance on low-bandwidth links when performing
bulk data reads. Of course I could also open separate pipes for
bulk and other, but this was quicker and less intrusive and doesn't
require authenticating twice.

branches: 1.10.4;
PUFFS_KFLAG_NOCACHE became two, so introduce the command line options
-o nonamecache and -o nopagecache and adapt file systems where necessary.

remove caveat no longer applicable

Add an EXAMPLES section.

Change [-O sshopt value] to [-O sshopt=value] to better conform with the
way other userland apps behave. In /etc/fstab one would specify options
with -O=sshopt=value.

Add flag [-O sshopt value] to mount_psshfs(8), allowing the user to specify
options to pass along to ssh(1). Remove redundant flag [-p sshport]; an
equivalent of -O Port sshport. ok'd by pooka@

Add -p to specify server port if someone should for some reason
happen to run sshd on port, to pick a random number, 443.

Probably should give a way to specify rest of the ssh options.

document -e for nfs exportability

* document -s
* explicitly mention that this implements sshfs functionality

New sentence, new line. Sort SEE ALSO. Punctuation nit.

man page for mount_psshfs

fix typos in word "successfully", mainly s/succesfully/successfully/.

PR/514612: Mateusz Paszwa: When no more data, bail out (reading from empty
files) instead of stuck looping.

branches: 1.64.22;
Ensure psshfs_node_read() completely reads all data before return.
Fixes file corruption in psshfs exposed with kernel MAXPHYS at 128KiB.

branches: 1.63.2;
Cache vattr in psshfs's setattr.

This means within the cache window, a setattr that wouldn't change the
remote file's attributes from our current view of them will not be
relayed to the server and wait for the server to answer. Thus, e.g., a
process with a periodic timer interrupt that calls open(2) in a loop
can make progress with much higher probability than without caching.

XXX The test case doesn't work, so it's currently disabled. It needs
to stop the child of sshd that is handling an sftp session, not sshd
itself, and it's not obvious how to do that.

ok pooka

Make error message more userfriendly in cases where server does
not support sftp.

In case we create a file, reclaim the vnode, and lookup the file
without the directory timeout expiring, we get vattr_null as the
attributes for that file. Ensure that we always report sane
attributes by getting them from the server if this is the case.
(also, sprinkle some const)

Fixes problem reported by dyoung. But wait! The bug's medallion
begins to glow! The bug looks much better! I crumble to dust.

There's probably another similar bug related to "lazy open". It
will trigger if we reclaim a node before the response to the open
arrives. Even the comments (typoless) know about this bug ...
But verifying it exists and fixing it will have to wait for another
day.

Add -u and -g, which allow to remap one (local,remote) uidgid, i.e.
umapfs without the fuss.

more lp64 it's-a-sign! fixes

Copyright year management. no functional change (unless sarah disagrees)

Introduce -c [12], which can be used to open 1 or 2 ssh connections
to the server. If "2" is specified, a separate connection is used
for data and directory operations. Using two connections can
significantly increase directory operation performance on a saturated
link, at least up to 30x faster.

Save errno before descending to cleanup gehennom.

branches: 1.55.2;
Track parent directory in cross-directory renames so that mv & rm
for the same file during a vnode's lifecycle works.

... rename fix .. it's going to be a good year ...

branches: 1.54.4;
void * -> puffs_cookie_t
no functional change

branches: 1.53.8;
When creating a node, do the server-dance before creating the local
data structure. This avoids a race between create and lookup, as
the kernel now unlock the directory before the create operation
returns.

Don't use puffs_cc_getusermount()

Get rid of a lot of extra complexity due to doing separate getattr
requests in readdir - just use the results from the readdir rpc.

In case a node is created in lookup in the readdir path, initialize
attribute cache timestamp to be equal to that of the directory
cache timestamp.

When scheduling waiters for re-run, remove them from the waitqueue
already when scheduling to make sure they're not scheduled more
than once.

Rototill.

Ok, ok, a few more words about it: stop holding puffs_cc as a holy
value and passing it around to almost every possible place (popquiz:
which kernel variable does this remind you of?). Instead, pass
the natural choice, puffs_usermount, and fetch puffs_cc via
puffs_cc_getcc() only in routines which actually need it. This
not only simplifies code, but (thanks to the introduction of
puffs_cc_getcc()) enables constructs which weren't previously sanely
possible, say layering as a curious example.

There's still a little to do on this front, but this was the major
fs interface blast.

Introduce the concept of lazyopens. This means that when a file
handle open is requested, it is waited for only if the node was
not previously succesfully opened. The actual wait for the file
handle happens only when the file handle is actually needed (read
or write). This in turn has the effect that reading cached files
will be quick instead of waiting for the file handle from the sftp
server first. The wait previously could be very long if there were
serveral hundred k of outstanding requests in a limited-bandwidth
link.

The code is in some need of serious handholding, but it works, so
I'll leave that as "future work".

Remove "puffs_cid" from the puffs interface following l-removal
from the kernel vfs interfaces. puffs_cc_getcaller(pcc) can be
used now should the same information be desired.

Don't bother tracking opencount and try to close file handles in
the close method - simply just wait for inactive and do it there.

Don't overload "attributes last read" for the symlink cache,
use a separate variable.

Cache readlink results similarly to attributes. Makes "cd" fast
for a slow connection if a symlink is along the path.

(this should be done elsewhere, but I want it for sshfs now)

Use puffs_cc_schedule() (which is an exported interface) instead of
plugging directly to puffs_goto().

Add option -r to specify maximum number of active read requests
for each node. Setting this to a small number can be used to
improve interactive performance on low-bandwidth links when performing
bulk data reads. Of course I could also open separate pipes for
bulk and other, but this was quicker and less intrusive and doesn't
require authenticating twice.

Close file handles if opencount drops to zero instead of closing
them if it does not drop to zero, i.e. fix a pretty obvious bug.

branches: 1.39.2;
If a node is forcibly reclaimed, close can be called more times
than open (or less, for the particular fs). So don't be uptight
about opencount.

Prevent crash in case there is more than one reader for the same
directory concurrently. Might need to revisit the method used here
later, though.

branches: 1.37.4;
make closehandles() void, because that's what it is

Track memory mappings. Close file handles already in close() if there
are no active mappings, otherwise do the standard inactive() run.

Oops, the cookies for nfs of course represent the *next* directory
entry offset, not the current one.

If the target node exists in rename, do not try to issue a setback
to signal no references, as that is not currently supported for
node_rename(). The removed node will not immediately be reclaimed,
but we can live with that for now.

While here, factor the removal code a bit to share with remove and
rmdir.

fixes PR kern/36637 by reinoud

adapt: inactive no longer comes with int *refcount (not that it was
used except for setting the default here anyway)

adapt: **newnode etc. pointers -> struct puffs_newinfo

adapt: pid -> const struct puffs_cid *

Move puffs to a two clause license where it already isn't so. And
as agc pointed out, even files with the third clause were already
effectively two clause because of a slight bug in the language...

remove obvious incorrect assert() from doreclaim() and remove the
whole PSN_NUKED bit - it's not very useful because of the noref setback.

* use NOREF setback to prompt immediate (well, once the kernel
reference count goes to 0) reclaim of deleted nodes as opposed to
waiting for the system to start reclaiming the freelist
* combine some nodeflags from different variable to one status variable

adapt to "step 3" of puffs_framebuf changes

adapt to puffs_framebuf changes

Use inactive-on-demand, set inactive request in open() to close
file handles when last reference is removed from the kernel.

Open files in open and close them in inactive. This avoids the
silly and problematic chmod + open + "un"chmod in write, among
other things.

Fire off FXP_CLOSE in create() and return immediately instead of
waiting for the result.

Adapt to use puffs_framebuf. This gets rid of most of the duplicated
code between mount_psshfs and mount_9p and clarifies the code
structure.

g/c forgotten printf

use puffs_cc_getspecific() where puffs_usermount is not required

use PUFFS_STORE_DCOOKIE

Actually, we can't reclaim our memory image for nodes which have
been nodetofh translated even if they are not valid on the sftp
server anymore, because some nfs client might still be clinging on
to the file handle we are reclaiming now.

Support nfs exporting.

Now, when I say support, I mean "support", due to the limitations
of the backend. File handles are valid only for one session, since
nodes can only be identified by pathnames and pathnames don't (all)
fit into the nfs file handle space. Additionally, we can't detect
if a pathname is completely replaced by another file (if it's done
via some other route that through our mount, of course). But then
again, that's an inherent problem with sshfs even without nfs.

Hide struct puffs_usermount from the rest of the world and provide
accessors for interesting data in it. Namely, you can now get
pu->pu_privdata with puffs_getspecific(), pu->pu_pn_root with
puffs_set/getroot() and pu->pu_maxreqlen with puffs_getmaxreqlen().

adapt to new readdir signature. no functional change

fix another case of checking for return value from wrong variable
(that's what you get when you copypaste code, a cid with a pin
to burst your bubble, that's what you get for all your troubles, I'll
never copypaste again)

CID 4461

Check for success from correct variable after allocnode()

CID 4460

If we have to do a stat in lookup, update time of last stat for
the new node to avoid a most likely immediate stat from the server.

In case lookup from the directory listing fails, try statting the
target before giving up. Makes it possible to access files from
-r+x directories.

* don't require a directory entry to exist for a reclaimee node.
this can happen legally when a file is removed from backing
storage not using this sshfs instance, a readdir is executed for
the parent directory and only then the node gets reclaimed.
* now that there is a mechanism in place which does not require a
pcc to do an sftp transaction, do not yield() in operations where
the final transaction is something where we don't care about the
return value (e.g. close handle). speedup benefit for no cost.

size_t -> uint32_t cleanup to make LP64 build

noticed by wiz

Support reclaim. To support dotdot lookup, here we leave the
reclaimed nodes hanging until all their children have been reclaimed
and then reclaim everything we can as far up to root as possible.
This is because the file system structures are currently interlinked
in a fashion which would make dotdot lookup based on purely a path
instead of a in-memory node parent member pointer very difficult.
Yes, this deserves a closer look some day.

Don't destroy puffs_nodes in success paths of remove and rmdir: the
kernel assumes that they will not be recycled until reclaim.

Implement "superreaddir". This issues a getattr for all the
directory entries already in readdir and caches the results instead
of waiting for each individial getattr from the kernel. For
high-latency links the difference in "ls -l" is quite astounding
and even on my lan "ls -lR" is faster than for nfs in a normal
directory hierarchy (i.e. not one artifically setup to have thousands
of files per directory).

TODO: implement some sort of bandwidth/latency measurement in the
code and enable or disable this option based on than information
(and a command-line flag).

* adapt to libpuffs path changes
* fix bug in rename

Give nukenode a destroy flag. If given, it completely wipes out
the node and directory cache (for directories). Otherwise it just
nukes directory entry information. The latter case is used when
renaming.

* Adapt to openssh standard of timestamps in file attributes. Maybe
should make this work with the IETF standard some day, also.
* kludge with writes and permissions a bit to be able to flush data
cached in ubc to files which are already with r/o permissions in
the backend

SUSv3 says: should the implementation not support directory unlinking,
unlink shall fail with EPERM and the errno of the failure shall be EPERM

puffs sshfs, an sshfs implementation built on top of the puffs
continuation framework

include the headers you use

branches: 1.18.6; 1.18.12;
Fix compilation with -Wsign-compare. Apparently 5.0 doesn't have
-Wsign-compare enabled, which explains a thing or two ...

pointed out by gson

Fix chgrp (and don't rely on VNOVAL being -1, however unlikely that
is to be broken).

Add -u and -g, which allow to remap one (local,remote) uidgid, i.e.
umapfs without the fuss.

more lp64 it's-a-sign! fixes

Copyright year management. no functional change (unless sarah disagrees)

Due to a recent encounter with a sucky internet connection, add
experimental option -p, which tries to reestablish the connection
to the sftp server in case it is lost. This currently has a few
limitations (found in the man page), but generally works in some
use cases.

Better support might eventually emerge, but since that requires a
plunge into the depths of puffs_framebuf, I need quite a bit of
Fernet Branca to build up my courage before attempting it.

branches: 1.12.8;
adapt to new cmpfb signature

branches: 1.11.4;
Fix return value comparison inversion. Makes psshfs work for the
nocache case again, as the server error value for EOF is properly
handled again.

Move puffs to a two clause license where it already isn't so. And
as agc pointed out, even files with the third clause were already
effectively two clause because of a slight bug in the language...

Change the signature of puffs_respcmp to return -1,0,1 for smaller,
equal, larger, respectively instead of 0/1 for non/equal. This
will allow sorting the buffers for faster matching in libpuffs.
While here, change the name from respcmp to framecmp, as that better
reflects the purpose.

NOTE! there is no obvious way to make compilation fail for file
systems which may already be using this feature (although I don't
think there are any outside our tree, as the feature is two weeks
old). Nevertheless, non-updated file systems will fail very quickly.

adapt to "step 3" of puffs_framebuf changes

uint32_t vs. size_t pointer adjustments for LP64 build

use puffs_framebuf_remaining()

Adapt to use puffs_framebuf. This gets rid of most of the duplicated
code between mount_psshfs and mount_9p and clarifies the code
structure.

Use ntohl instead of htonl when converting data from the network.
Result isn't that much different, though ....

size_t -> uint32_t cleanup to make LP64 build

noticed by wiz

* Adapt to openssh standard of timestamps in file attributes. Maybe
should make this work with the IETF standard some day, also.
* kludge with writes and permissions a bit to be able to flush data
cached in ubc to files which are already with r/o permissions in
the backend

puffs sshfs, an sshfs implementation built on top of the puffs
continuation framework

s/systme/system/ in comment.

include the headers you use

branches: 1.65.2; 1.65.8;
Use __dead

Bring mount_psshfs user/host/path argument processing in line with sftp(1).

Initialize psshfs root node's readdir waiters queue.

Every other node's readder waiters queue is initialized in makenode,
but the root node is created specially.

ok pooka

Make error message more userfriendly in cases where server does
not support sftp.

umgah. meant fts, not fsu

Bump the initial rootdir link count guess to a ludicrous value.

Make r/w servers unmount themselves cleanly upon receiving SIGINT/TERM.

errx() already contains progname ...

Don't allow mangling of uid or gid -1, since they have a special
meaning in the sftp protocol.

Add -u and -g, which allow to remap one (local,remote) uidgid, i.e.
umapfs without the fuss.

const woes

wait() for dead ssh processes (but don't block, so we might have
one teeny zmobie in the system).

Fix SNAFU in previous: connection handshake is already handled by
the connect routine now, so we're not required/allowed to do it twice.
Makes reconnect on dead link work again.

Copyright year management. no functional change (unless sarah disagrees)

Introduce -c [12], which can be used to open 1 or 2 ssh connections
to the server. If "2" is specified, a separate connection is used
for data and directory operations. Using two connections can
significantly increase directory operation performance on a saturated
link, at least up to 30x faster.

Support statvfs@openssh.com extension where available. Makes df(1)
return something other than just 0.
caveat: statvfs is done for the mountpoint path, so might not give
the truth about a directory inside the mountpoint.

branches: 1.49.6;
Due to a recent encounter with a sucky internet connection, add
experimental option -p, which tries to reestablish the connection
to the sftp server in case it is lost. This currently has a few
limitations (found in the man page), but generally works in some
use cases.

Better support might eventually emerge, but since that requires a
plunge into the depths of puffs_framebuf, I need quite a bit of
Fernet Branca to build up my courage before attempting it.

call puffs_exit()

branches: 1.47.6;
Add a -F option to allow passing a configuration file to ssh(1), instead
of having to use multiple -O options on the command line. OKed by pooka@.

Introduce the concept of lazyopens. This means that when a file
handle open is requested, it is waited for only if the node was
not previously succesfully opened. The actual wait for the file
handle happens only when the file handle is actually needed (read
or write). This in turn has the effect that reading cached files
will be quick instead of waiting for the file handle from the sftp
server first. The wait previously could be very long if there were
serveral hundred k of outstanding requests in a limited-bandwidth
link.

The code is in some need of serious handholding, but it works, so
I'll leave that as "future work".

Don't bother tracking opencount and try to close file handles in
the close method - simply just wait for inactive and do it there.

use puffs_daemon() instead of daemon()

Don't overload "attributes last read" for the symlink cache,
use a separate variable.

Make it possible to force attribute/directory cache re-read by
sending SIGHUP to mount_psshfs.

don't allow negative timeouts apart from -1

Make attribute & directory content caching timeout a command line
option. Use -t -1 to never expire, which is nifty on a high-latency
link where you know the server won't be modified from under you.

Open the kernel descriptor as part of mount(), not init(). Then
it doesn't matter if someone fork()s or does other tricks between
init() and mount() (and besides, now it's where it logically should be).

Actually, daemonize the file servers before mounting. I might require
at some point that the local protocol handler does not change after
the file system has been mounted.

Pull the daemonizing code out of the library mainloop into the file
servers. Calling daemon() (i.e. fork()ing) inside a library can
cause nice surprises for e.g. threaded programs. As discussed with
Greg Oster & others.

Add option -r to specify maximum number of active read requests
for each node. Setting this to a small number can be used to
improve interactive performance on low-bandwidth links when performing
bulk data reads. Of course I could also open separate pipes for
bulk and other, but this was quicker and less intrusive and doesn't
require authenticating twice.

Adapt to new puffs_framev_init() signature - gotfb.

branches: 1.34.4; 1.34.6;
Track memory mappings. Close file handles already in close() if there
are no active mappings, otherwise do the standard inactive() run.

* add mntfromname parameter to puffs_init()
* set it in file servers

I'm fairly certain this file system's type is psshfs, not ppshfs
(how did that manage to be there this long?)

adapt to puffs_framebuf changes

g/c bit about MNT_NOATIME being mandatory. it obvious isn't since
the code was #if 0'd out.

Move puffs to a two clause license where it already isn't so. And
as agc pointed out, even files with the third clause were already
effectively two clause because of a slight bug in the language...

devnullify ssh process's stderr to avoid tty spam

reflect changes in fs mount: use puffs_mount() instead of
puffs_domount() & puffs_start()

adapt to "step 3" of puffs_framebuf changes

use puffs_mainloop(); puffs_framebuf_eventloop() is a goner

adapt to puffs_framebuf changes

use fprintf instead of errx for usage()

Change [-O sshopt value] to [-O sshopt=value] to better conform with the
way other userland apps behave. In /etc/fstab one would specify options
with -O=sshopt=value.

Add flag [-O sshopt value] to mount_psshfs(8), allowing the user to specify
options to pass along to ssh(1). Remove redundant flag [-p sshport]; an
equivalent of -O Port sshport. ok'd by pooka@

Use inactive-on-demand, set inactive request in open() to close
file handles when last reference is removed from the kernel.

Open files in open and close them in inactive. This avoids the
silly and problematic chmod + open + "un"chmod in write, among
other things.

adapt to puffs_framebuf_eventloop signature change

Adapt to use puffs_framebuf. This gets rid of most of the duplicated
code between mount_psshfs and mount_9p and clarifies the code
structure.

For consistency destroy kernel request putbuffers also when
escaping event loop (the process is going to exit anyway, but ...)

Add -p to specify server port if someone should for some reason
happen to run sshd on port, to pick a random number, 443.

Probably should give a way to specify rest of the ssh options.

Use PUFFS_KFLAG_WTCACHE to avoid the need to do a sync(8) on the
client host to get all the data onto the server.

adapt to kernel changes with file handles

Instead of keeping on adding parameters to puffs_mount(), make it
only take the bare essentials, which currently means removing
"maxreqlen" from the argument list (all current callers I'm aware
of set it as 0 anyway). Introduce puffs_init(), which provides a
context for setting various parameters and puffs_domount(), which
can be used to mount the file system. Keep puffs_mount() as a
shortcut for the above two for simple file systems.

Bump development ABI version to 13. After all, it's Friday the 13th.
Watch out! Bad things can happen on Friday the 13th. --No carrier--

Support nfs exporting.

Now, when I say support, I mean "support", due to the limitations
of the backend. File handles are valid only for one session, since
nodes can only be identified by pathnames and pathnames don't (all)
fit into the nfs file handle space. Additionally, we can't detect
if a pathname is completely replaced by another file (if it's done
via some other route that through our mount, of course). But then
again, that's an inherent problem with sshfs even without nfs.

Hide struct puffs_usermount from the rest of the world and provide
accessors for interesting data in it. Namely, you can now get
pu->pu_privdata with puffs_getspecific(), pu->pu_pn_root with
puffs_set/getroot() and pu->pu_maxreqlen with puffs_getmaxreqlen().

* don't require a directory entry to exist for a reclaimee node.
this can happen legally when a file is removed from backing
storage not using this sshfs instance, a readdir is executed for
the parent directory and only then the node gets reclaimed.
* now that there is a mechanism in place which does not require a
pcc to do an sftp transaction, do not yield() in operations where
the final transaction is something where we don't care about the
return value (e.g. close handle). speedup benefit for no cost.

Support reclaim. To support dotdot lookup, here we leave the
reclaimed nodes hanging until all their children have been reclaimed
and then reclaim everything we can as far up to root as possible.
This is because the file system structures are currently interlinked
in a fashion which would make dotdot lookup based on purely a path
instead of a in-memory node parent member pointer very difficult.
Yes, this deserves a closer look some day.

Implement "superreaddir". This issues a getattr for all the
directory entries already in readdir and caches the results instead
of waiting for each individial getattr from the kernel. For
high-latency links the difference in "ls -l" is quite astounding
and even on my lan "ls -lR" is faster than for nfs in a normal
directory hierarchy (i.e. not one artifically setup to have thousands
of files per directory).

TODO: implement some sort of bandwidth/latency measurement in the
code and enable or disable this option based on than information
(and a command-line flag).

more ketchup

play catchup with the lib

don't detach if -s or -o dump is given on the command line

use UBC by default

* do mount as a forward operation instead of a callback
* process -o args for mount

puffs sshfs, an sshfs implementation built on top of the puffs
continuation framework

In case we create a file, reclaim the vnode, and lookup the file
without the directory timeout expiring, we get vattr_null as the
attributes for that file. Ensure that we always report sane
attributes by getting them from the server if this is the case.
(also, sprinkle some const)

Fixes problem reported by dyoung. But wait! The bug's medallion
begins to glow! The bug looks much better! I crumble to dust.

There's probably another similar bug related to "lazy open". It
will trigger if we reclaim a node before the response to the open
arrives. Even the comments (typoless) know about this bug ...
But verifying it exists and fixing it will have to wait for another
day.

Add -u and -g, which allow to remap one (local,remote) uidgid, i.e.
umapfs without the fuss.

more lp64 it's-a-sign! fixes

Copyright year management. no functional change (unless sarah disagrees)

Introduce -c [12], which can be used to open 1 or 2 ssh connections
to the server. If "2" is specified, a separate connection is used
for data and directory operations. Using two connections can
significantly increase directory operation performance on a saturated
link, at least up to 30x faster.

Support statvfs@openssh.com extension where available. Makes df(1)
return something other than just 0.
caveat: statvfs is done for the mountpoint path, so might not give
the truth about a directory inside the mountpoint.

branches: 1.34.6;
Due to a recent encounter with a sucky internet connection, add
experimental option -p, which tries to reestablish the connection
to the sftp server in case it is lost. This currently has a few
limitations (found in the man page), but generally works in some
use cases.

Better support might eventually emerge, but since that requires a
plunge into the depths of puffs_framebuf, I need quite a bit of
Fernet Branca to build up my courage before attempting it.

branches: 1.33.8;
Don't use puffs_cc_getusermount()

Get rid of a lot of extra complexity due to doing separate getattr
requests in readdir - just use the results from the readdir rpc.

Rototill.

Ok, ok, a few more words about it: stop holding puffs_cc as a holy
value and passing it around to almost every possible place (popquiz:
which kernel variable does this remind you of?). Instead, pass
the natural choice, puffs_usermount, and fetch puffs_cc via
puffs_cc_getcc() only in routines which actually need it. This
not only simplifies code, but (thanks to the introduction of
puffs_cc_getcc()) enables constructs which weren't previously sanely
possible, say layering as a curious example.

There's still a little to do on this front, but this was the major
fs interface blast.

Introduce the concept of lazyopens. This means that when a file
handle open is requested, it is waited for only if the node was
not previously succesfully opened. The actual wait for the file
handle happens only when the file handle is actually needed (read
or write). This in turn has the effect that reading cached files
will be quick instead of waiting for the file handle from the sftp
server first. The wait previously could be very long if there were
serveral hundred k of outstanding requests in a limited-bandwidth
link.

The code is in some need of serious handholding, but it works, so
I'll leave that as "future work".

Don't bother tracking opencount and try to close file handles in
the close method - simply just wait for inactive and do it there.

Don't overload "attributes last read" for the symlink cache,
use a separate variable.

Cache readlink results similarly to attributes. Makes "cd" fast
for a slow connection if a symlink is along the path.

(this should be done elsewhere, but I want it for sshfs now)

Make attribute & directory content caching timeout a command line
option. Use -t -1 to never expire, which is nifty on a high-latency
link where you know the server won't be modified from under you.

Add option -r to specify maximum number of active read requests
for each node. Setting this to a small number can be used to
improve interactive performance on low-bandwidth links when performing
bulk data reads. Of course I could also open separate pipes for
bulk and other, but this was quicker and less intrusive and doesn't
require authenticating twice.

adapt to new cmpfb signature

branches: 1.23.2;
Prevent crash in case there is more than one reader for the same
directory concurrently. Might need to revisit the method used here
later, though.

branches: 1.22.4;
Track memory mappings. Close file handles already in close() if there
are no active mappings, otherwise do the standard inactive() run.

adapt to puffs_framebuf changes

Move puffs to a two clause license where it already isn't so. And
as agc pointed out, even files with the third clause were already
effectively two clause because of a slight bug in the language...

Observe that when doing ls -l, the attributes of the first file
getattr are usually still outstanding when we already would like
the result. Instead of issueing another stat which will be serviced
only after all the other entries in the directory, record all the
outgoing readdir getattr buffers and if we encounter an outstanding
request when we need to fetch attrs, do a puffs_framev_framebuf_ccpromote()
wait for it instead of firing off the second query. This shaves
almost 10% off the time for ls -lR.

Also, get rid of the SUPERREADDIR conditional, since it has penetrated
the code quite a bit and the #ifdef SUPERREADDIRs were starting to
look like tagliatelle alla bolognese (n.b. I love how it looks,
but I wouldn't like it either if my tagliatelle alla bolognese
looked like psshfs code). Maybe it should be re-introduced in the
form of a switch?

Implement SUPERREADDIR (like nfs readdirplus) getattr cache warming
a bit differently: when reading the directory, store all getattr
caching queries and fire off only when the directory read is
complete. That way the common sequence is not [readdir, lots of
async getattr requests, readdir EOF] but rather [readdir, readdir
EOF, lots of async getattr]. This speeds up ls -lR by about 25%
(on my LAN).

remove obvious incorrect assert() from doreclaim() and remove the
whole PSN_NUKED bit - it's not very useful because of the noref setback.

* use NOREF setback to prompt immediate (well, once the kernel
reference count goes to 0) reclaim of deleted nodes as opposed to
waiting for the system to start reclaiming the freelist
* combine some nodeflags from different variable to one status variable

adapt to "step 3" of puffs_framebuf changes

uint32_t vs. size_t pointer adjustments for LP64 build

adapt to puffs_framebuf changes

Open files in open and close them in inactive. This avoids the
silly and problematic chmod + open + "un"chmod in write, among
other things.

Adapt to use puffs_framebuf. This gets rid of most of the duplicated
code between mount_psshfs and mount_9p and clarifies the code
structure.

use puffs_cc_getspecific() where puffs_usermount is not required

Support nfs exporting.

Now, when I say support, I mean "support", due to the limitations
of the backend. File handles are valid only for one session, since
nodes can only be identified by pathnames and pathnames don't (all)
fit into the nfs file handle space. Additionally, we can't detect
if a pathname is completely replaced by another file (if it's done
via some other route that through our mount, of course). But then
again, that's an inherent problem with sshfs even without nfs.

Hide struct puffs_usermount from the rest of the world and provide
accessors for interesting data in it. Namely, you can now get
pu->pu_privdata with puffs_getspecific(), pu->pu_pn_root with
puffs_set/getroot() and pu->pu_maxreqlen with puffs_getmaxreqlen().

In case lookup from the directory listing fails, try statting the
target before giving up. Makes it possible to access files from
-r+x directories.

size_t -> uint32_t cleanup to make LP64 build

noticed by wiz

Support reclaim. To support dotdot lookup, here we leave the
reclaimed nodes hanging until all their children have been reclaimed
and then reclaim everything we can as far up to root as possible.
This is because the file system structures are currently interlinked
in a fashion which would make dotdot lookup based on purely a path
instead of a in-memory node parent member pointer very difficult.
Yes, this deserves a closer look some day.

Implement "superreaddir". This issues a getattr for all the
directory entries already in readdir and caches the results instead
of waiting for each individial getattr from the kernel. For
high-latency links the difference in "ls -l" is quite astounding
and even on my lan "ls -lR" is faster than for nfs in a normal
directory hierarchy (i.e. not one artifically setup to have thousands
of files per directory).

TODO: implement some sort of bandwidth/latency measurement in the
code and enable or disable this option based on than information
(and a command-line flag).

* adapt to libpuffs path changes
* fix bug in rename

* do mount as a forward operation instead of a callback
* process -o args for mount

puffs sshfs, an sshfs implementation built on top of the puffs
continuation framework

Move puffs to a two clause license where it already isn't so. And
as agc pointed out, even files with the third clause were already
effectively two clause because of a slight bug in the language...

puffs sshfs, an sshfs implementation built on top of the puffs
continuation framework

include the headers you use

branches: 1.50.6; 1.50.12;
In case we create a file, reclaim the vnode, and lookup the file
without the directory timeout expiring, we get vattr_null as the
attributes for that file. Ensure that we always report sane
attributes by getting them from the server if this is the case.
(also, sprinkle some const)

Fixes problem reported by dyoung. But wait! The bug's medallion
begins to glow! The bug looks much better! I crumble to dust.

There's probably another similar bug related to "lazy open". It
will trigger if we reclaim a node before the response to the open
arrives. Even the comments (typoless) know about this bug ...
But verifying it exists and fixing it will have to wait for another
day.

Save "." attributes in readdir. This fixes the root link count
after readdir is called for the root dir the first time (yes,
there's still a window of wrong link count after the fs is mounted.
it's currently quite difficult to call sftp_readdir() from outside
the main loop).

Should fix "find /mnt" problem for a mountpoint with more than the
"guessed" amount of subdirectories, as reported by dyoung.

Add -u and -g, which allow to remap one (local,remote) uidgid, i.e.
umapfs without the fuss.

more lp64 it's-a-sign! fixes

Introduce -c [12], which can be used to open 1 or 2 ssh connections
to the server. If "2" is specified, a separate connection is used
for data and directory operations. Using two connections can
significantly increase directory operation performance on a saturated
link, at least up to 30x faster.

Record inode number in psshfs_dir also for already instantiated
nodes when doing readdir. This makes pwd work again for cases
where getcwd() actually has to do the "READDIR + compare inode
numbers" trick.

Yet another problem reported by jmmv.

*blink*, remember to update comment too

Do previous in a way which causes less waste of potential inode numbers.

Fix lossage noticed by jmmv (mostly for -t 0 mounts) and make sure
a node always has the inode number set. And since I'm feeling
generous, sprinkle a few comments around the affected areas (mostly
so that I'd remember what in the world the code is trying to do).

Don't use puffs_cc_getusermount()

Get rid of a lot of extra complexity due to doing separate getattr
requests in readdir - just use the results from the readdir rpc.

Invalidate namecache for a directory only if we've already read it
at least once.

Rototill.

Ok, ok, a few more words about it: stop holding puffs_cc as a holy
value and passing it around to almost every possible place (popquiz:
which kernel variable does this remind you of?). Instead, pass
the natural choice, puffs_usermount, and fetch puffs_cc via
puffs_cc_getcc() only in routines which actually need it. This
not only simplifies code, but (thanks to the introduction of
puffs_cc_getcc()) enables constructs which weren't previously sanely
possible, say layering as a curious example.

There's still a little to do on this front, but this was the major
fs interface blast.

Introduce the concept of lazyopens. This means that when a file
handle open is requested, it is waited for only if the node was
not previously succesfully opened. The actual wait for the file
handle happens only when the file handle is actually needed (read
or write). This in turn has the effect that reading cached files
will be quick instead of waiting for the file handle from the sftp
server first. The wait previously could be very long if there were
serveral hundred k of outstanding requests in a limited-bandwidth
link.

The code is in some need of serious handholding, but it works, so
I'll leave that as "future work".

If a regular file was remotely updated and a readdir was done on
the directory which contained the file before a getattr on the file
itself, the locally cached mtime would be updated without invalidating
the kernel page cache. Thus incorrect data would be returned when
the node was read afterwards as the node size wouldn't match the
data length in the page cache.

Fix the problem by making all vattr-setting routines use the same code.

Problem noticed again by jmmv & atf (and again by running atf over
psshfs ... sometimes you're the windshield, sometimes you're the bug)

Drop trailing whitespace. On behalf of pooka.

When checking if the file page cache should be invalidated, compare
against cached mtime instead of attrread - attrread can be reset
these days by sending SIGHUP.

Problem noticed by jmmv & atf (well.. namely by using atf through psshfs).

Cache readlink results similarly to attributes. Makes "cd" fast
for a slow connection if a symlink is along the path.

(this should be done elsewhere, but I want it for sshfs now)

Make it possible to force attribute/directory cache re-read by
sending SIGHUP to mount_psshfs.

Make attribute & directory content caching timeout a command line
option. Use -t -1 to never expire, which is nifty on a high-latency
link where you know the server won't be modified from under you.

Add option -r to specify maximum number of active read requests
for each node. Setting this to a small number can be used to
improve interactive performance on low-bandwidth links when performing
bulk data reads. Of course I could also open separate pipes for
bulk and other, but this was quicker and less intrusive and doesn't
require authenticating twice.

Free longname in all error branches. CID: 4545

branches: 1.28.2;
error branch management. CID 4503

jump to correct error branch. CID 4500

Prevent crash in case there is more than one reader for the same
directory concurrently. Might need to revisit the method used here
later, though.

print warning if directory cache invalidation fails

Move puffs to a two clause license where it already isn't so. And
as agc pointed out, even files with the third clause were already
effectively two clause because of a slight bug in the language...

In case we are really tight on vnodes, the directory vnode might
be reclaimed from under while we are warming the getattr cache.
Shuffle some code to prevent the effects. Theoretically the race
is still possible, but I don't think it will happen in practice.
In any case, the code could benefit from some more dusting.

Observe that when doing ls -l, the attributes of the first file
getattr are usually still outstanding when we already would like
the result. Instead of issueing another stat which will be serviced
only after all the other entries in the directory, record all the
outgoing readdir getattr buffers and if we encounter an outstanding
request when we need to fetch attrs, do a puffs_framev_framebuf_ccpromote()
wait for it instead of firing off the second query. This shaves
almost 10% off the time for ls -lR.

Also, get rid of the SUPERREADDIR conditional, since it has penetrated
the code quite a bit and the #ifdef SUPERREADDIRs were starting to
look like tagliatelle alla bolognese (n.b. I love how it looks,
but I wouldn't like it either if my tagliatelle alla bolognese
looked like psshfs code). Maybe it should be re-introduced in the
form of a switch?

Implement SUPERREADDIR (like nfs readdirplus) getattr cache warming
a bit differently: when reading the directory, store all getattr
caching queries and fire off only when the directory read is
complete. That way the common sequence is not [readdir, lots of
async getattr requests, readdir EOF] but rather [readdir, readdir
EOF, lots of async getattr]. This speeds up ls -lR by about 25%
(on my LAN).

remove obvious incorrect assert() from doreclaim() and remove the
whole PSN_NUKED bit - it's not very useful because of the noref setback.

* use NOREF setback to prompt immediate (well, once the kernel
reference count goes to 0) reclaim of deleted nodes as opposed to
waiting for the system to start reclaiming the freelist
* combine some nodeflags from different variable to one status variable

Add error parameter to puffs_framev_cb to signal if there is a
result of if the function should just do resource cleanup.

remember to free longname if getting vattr from the message fails

CID 4486

adapt to "step 3" of puffs_framebuf changes

adapt to puffs_framebuf changes

Adapt to use puffs_framebuf. This gets rid of most of the duplicated
code between mount_psshfs and mount_9p and clarifies the code
structure.

Hide struct puffs_usermount from the rest of the world and provide
accessors for interesting data in it. Namely, you can now get
pu->pu_privdata with puffs_getspecific(), pu->pu_pn_root with
puffs_set/getroot() and pu->pu_maxreqlen with puffs_getmaxreqlen().

In case lookup from the directory listing fails, try statting the
target before giving up. Makes it possible to access files from
-r+x directories.

* don't require a directory entry to exist for a reclaimee node.
this can happen legally when a file is removed from backing
storage not using this sshfs instance, a readdir is executed for
the parent directory and only then the node gets reclaimed.
* now that there is a mechanism in place which does not require a
pcc to do an sftp transaction, do not yield() in operations where
the final transaction is something where we don't care about the
return value (e.g. close handle). speedup benefit for no cost.

one more size_t -> uint32_t

Support reclaim. To support dotdot lookup, here we leave the
reclaimed nodes hanging until all their children have been reclaimed
and then reclaim everything we can as far up to root as possible.
This is because the file system structures are currently interlinked
in a fashion which would make dotdot lookup based on purely a path
instead of a in-memory node parent member pointer very difficult.
Yes, this deserves a closer look some day.

Implement "superreaddir". This issues a getattr for all the
directory entries already in readdir and caches the results instead
of waiting for each individial getattr from the kernel. For
high-latency links the difference in "ls -l" is quite astounding
and even on my lan "ls -lR" is faster than for nfs in a normal
directory hierarchy (i.e. not one artifically setup to have thousands
of files per directory).

TODO: implement some sort of bandwidth/latency measurement in the
code and enable or disable this option based on than information
(and a command-line flag).

* adapt to libpuffs path changes
* fix bug in rename

Give nukenode a destroy flag. If given, it completely wipes out
the node and directory cache (for directories). Otherwise it just
nukes directory entry information. The latter case is used when
renaming.

Invalidate kernel name cache for a directory when (re-)reading
directory contents from the server.

Mark invalid attributes as invalid. Fixes stuff such a cp'ing files
when cp uses mmap(). Also makes attributes appear correct.

properly carry fileid for directory entries, fixes pwd

fix link count calculation for directories

puffs sshfs, an sshfs implementation built on top of the puffs
continuation framework