Home | History | Annotate | Download | only in puffs
History log of /src/sys/fs/puffs/puffs_sys.h
RevisionDateAuthorComments
 1.91  27-Jan-2019  pgoyette Merge the [pgoyette-compat] branch
 1.90  07-Jul-2016  msaitoh branches: 1.90.16; 1.90.18;
KNF. Remove extra spaces. No functional change.
 1.89  15-Feb-2015  manu Add PUFFS_KFLAG_NOFLUSH_META to prevent sending metadata flush to FUSE

FUSE filesystems do not expect to get metadata updates for [amc]time
and size, they updates the value on their own after operations.

The PUFFS PUFFS_KFLAG_NOFLUSH_META option prevents regular metadata cache
flushes to the filesystem , and libperfuse uses it to match Linux FUSE
behavior.

While there, fix a bug in SETATTR: do not update kernel metadata cache
from SETATTR reply when the request is asynchronous, as we do not have
the reply yet.
 1.88  05-Oct-2014  apb branches: 1.88.2;
Add close brace, accidentally omitted from previous change.
 1.87  05-Oct-2014  apb Safer definitions of DPRINTF and DPRINTF_VERBOSE.

In the PUFFSDEBUG case, wrap do { ... } while (/*CONSTCOND*/0)
around the definitions. In the non-PUFFSDEBUG case, define them
as ((void)0) instead of as empty.
 1.86  28-Aug-2014  hannken Change puffs from hashlist to vcache.
- field "pa_nhashbuckets" of struct "puffs_kargs" becomes a no-op.
and should be removed on the next protocol version bump.
 1.85  16-Aug-2014  manu Add a oflags input field to open requests so that the filesystem can pass
back information about the file. Implement PUFFS_OPEN_IO_DIRECT, which
will force direct IO (bypassing page cache) for the file.
 1.84  17-Oct-2013  christos branches: 1.84.4;
- remove unused variables
- add _NOERROR flavor macros for the case where errors are ignored.
 1.83  06-Mar-2013  yamt branches: 1.83.6;
comment
 1.82  11-Aug-2012  manu branches: 1.82.2;
Missing bit in previous commit (PUFFS_KFLAG_CACHE_DOTDOT option to avoid
looking up ..)
 1.81  27-Jul-2012  manu Rename slow sopreq queue into node sopreq queue, to refet the fact that
is only intended for postponed node reclaims.
When purging the node sopreq queue, do not call puffs_msg_sendresp(), as
it makes no sense.
 1.80  21-Jul-2012  manu - Improve PUFFS_KFLAG_CACHE_FS_TTL by reclaiming older inactive nodes.

The normal kernel behavior is to retain inactive nodes in the freelist
until it runs out of vnodes. This has some merit for local filesystems,
where the cost of an allocation is about the same as the cost of a
lookup. But that situation is not true for distributed filesystems.
On the other hand, keeping inactive nodes for a long time hold memory
in the file server process, and when the kernel runs out of vnodes, it
produce reclaim avalanches that increase lattency for other operations.

We do not reclaim inactive vnodes immediatly either, as they may be
looked up again shortly. Instead we introduce a grace time and we
reclaim nodes that have been inactive beyond the grace time.

- Fix lookup/reclaim race condition.

The above improvement undercovered a race condition between lookup and
reclaim. If we reclaimed a vnode associated with a userland cookie while
a lookup returning that same cookiewas inprogress, then the kernel ends
up with a vnode associated with a cookie that has been reclaimed in
userland. Next operation on the cookie will crash (or at least confuse)
the filesystem.

We fix this by introducing a lookup count in kernel and userland. On
reclaim, the kernel sends the count, which enable userland to detect
situation where it initiated a lookup that is not completed in kernel.
In such a situation, the reclaim must be ignored, as the node is about
to be looked up again.
 1.79  08-Apr-2012  manu Add name and atttribute cache with filesytem provided TTL.
lookup, create, mknod, mkdir, symlink, getattr and setattr messages
have been extended so that attributes and their TTL can be provided
by the filesytem. lookup, create, mknod, mkdir, and symlink messages
are also extended so that the filesystem can provide name TTL.
 1.78  29-Aug-2011  manu branches: 1.78.2; 1.78.6; 1.78.8;
Add a mutex for operations that touch size (setattr, getattr, write, fsync).

This is required to avoid data corruption bugs, where a getattr slices
itself within a setattr operation, and sets the size to the stall value
it got from the filesystem. That value is smaller than the one set by
setattr, and the call to uvm_vnp_setsize() trigged a spurious truncate.
The result is a chunk of zeroed data in the file.

Such a situation can easily happen when the ioflush thread issue a
VOP_FSYNC/puffs_vnop_sync/flushvncache/dosetattrn while andother process
do a sys_stat/VOP_GETATTR/puffs_vnop_getattr.

This mutex on size operation can be removed the day we decide VOP_GETATTR
has to operated on a locked vnode, since the other operations that touch
size already require that.
 1.77  11-Jan-2011  kefren add advlock to puffs. ok pooka@
should fix kern/43321
 1.76  06-Jul-2010  pooka Add compat to enable running puffs in a 64bit time_t kernel against
a server which runs in 32bit time_t namespace.
 1.75  07-Jan-2010  pooka branches: 1.75.2; 1.75.4;
Rename PUFFS_SOPREQ_EXIT to PUFFS_SOPREQSYS_EXIT to better signal
it comes from within the kernel instead of as a direct result of
a user request.

no functional change
 1.74  07-Jan-2010  pooka Add a PUFFS_UNMOUNT server->kernel request, which causes the kernel
to initiate self destruct, i.e. unmount(MNT_FORCE). This, however,
is a semi-controlled self-destruct, since all caches are flushed
before the (possibly) violent unmount takes place.
 1.73  07-Dec-2009  pooka Process flush requests from the file server in a separate thread
context. This fixes a long-standing but seldomly seen deadlock,
where the kernel was holding pages busy (due to e.g. readahead
request) while waiting for the server to respond, and the server
made a callback into the kernel asking to invalidate those pages.
... or, well, theoretically fixes, since I didn't have any reliable
way of repeating the deadlock and I think I saw it only twice.
 1.72  05-Nov-2009  pooka Kill suspend support. It was never implemented correctly:
* it depended on the biglock (in a very cruel way)
* it was attached to userspace transactions rather than logical
fs operations

(If someone wants to revisit it some day, most of the stuff can be
reused from cvs history)
 1.71  05-Nov-2009  pooka Reinstante PNODE_DYING. vmlocking had a brief hiatus when it was not
a valid optimization, but that's long gone and once VOP_INACTIVE is
called and the file server says that the vnode is going to be recycled,
it really is going to be recycled extra references gained or not.
 1.70  28-Jan-2008  pooka branches: 1.70.10; 1.70.20; 1.70.28;
For code clarity typedef void *puffs_cookie_t.

No functional change.
 1.69  02-Jan-2008  pooka More type-punning workarounds. Curiously the kernel compilation
flags cause gcc to not complain.
 1.68  02-Jan-2008  ad Merge vmlocking2 to head.
 1.67  08-Dec-2007  pooka branches: 1.67.4;
Now that "l" is gone both as an argument to operations and from
componentname, remove all vestiges of puffs_cid.
 1.66  05-Dec-2007  pooka Send a response message for flush operations from the kernel instead
of abusing the return value of write(2).
 1.65  20-Nov-2007  pooka branches: 1.65.2;
Retire M_PUFFS, use kmem(9) instead.
 1.64  17-Nov-2007  pooka Make puffs_updatenode() take a puffs_node instead of a vnode. This
way we don't need to worry if a vnode has been reclaimed from under
us.
 1.63  17-Nov-2007  pooka Implement a biodone callback for async writes similar to reads and
use that when possible.
 1.62  16-Nov-2007  pooka Restructure the messaging interface a bit more: make all interfacing
with the file server happen through puffs_msg_enqueue() and
puffs_msg_wait() instead of having a billion different routines.
Build the existing system upon these two. Most importantly though,
decouple insertation into the op queue from the actual wait. This
is useful for a number of reasons coming soon to a cvs repo near you.
 1.61  12-Nov-2007  pooka Bounds-check responses from userspace.
 1.60  10-Nov-2007  pooka Part 2/n of extensive changes to request transport to/from userspace:

Rip the transport code completely out of puffs and generalize it
into an independent module which will be used for multiple purposes
in the future. This module is called the Pass-to-Userspace
Transporter (known as "putter" among friends).

This is very much work-in-progress and one dependency with puffs
remains: the request framing format.

The device name is still /dev/puffs, but that will change soon.

Users of puffs need the following in their kernel configs now:
pseudo-device putter
 1.59  11-Oct-2007  pooka branches: 1.59.2; 1.59.4;
Part 1/n of some pretty extensive changes to how the kernel module
interacts with the userspace file server:

* since the kernel-user communication is not purely request-response
anymore (hasn't been since 2006), try to rename some "request" to
"message". more similar mangling will take place in the future.

* completely rework how messages are allocated. previously most of
them were borrowed from the stack (originally *all* of them),
but now always allocate dynamically. this makes the structure
of the code much cleaner. also makes it possible to fix a
locking order violation. it enables plenty of future enhancements.

* start generalizing the transport interface to be independent of puffs

* move transport interface to read/write instead of ioctl. the
old one had legacy design problems, and besides, ioctl's suck.
implement a very generic version for now; this will be
worked on later hopefully some day reaching "highly optimized".

* implement libpuffs support behind existing library request
interfaces. this will change eventually (I hate those interfaces)
 1.58  09-Oct-2007  pooka g/c more unused stuff
 1.57  09-Oct-2007  pooka g/c vntouser_req(), it's not used anymore
 1.56  04-Oct-2007  pooka g/c the "sizeop" code previous used for ioctl/fcntl. It was already
commented out and has bitrotted beyond all recognition, so it needs
complete rethinking.
 1.55  02-Oct-2007  pooka If kernel resource allocation fails after the file server has
committed something, issue an abort. The abort is done through
the regular op channel, e.g. failed mkdir leads to regular rmdir,
inactive and reclaim. No internal interface is planned currently
for the one file system out of a million which would implement it
to benefit from the one case in a billion where kernel resource
allocation actually does fail and out of that one case in a trillion
where internal vs. external would make a difference.
 1.54  01-Oct-2007  pooka * better error checking: validate error values received from userland
to be vaild errno values
* include string describing error in PUFFS_ERR
* get rid of union in puffs_req, it's nothing but trouble
* pass pmp to async i/o callbacks
 1.53  27-Sep-2007  pooka Split routines handling nodes from puffs_subr to puffs_node.
No functional change.
 1.52  27-Sep-2007  pooka Differentiate between cookie2vnode returning an error and
return to caller, address unknown: no such cookie, no such node.
Make the callers use this info to either create a new vnode or bail.
 1.51  27-Sep-2007  pooka Add error notifications, which are used to deliver errors from the
kernel to the file server for silly things the file server did,
e.g. attempting to create a file with size VSIZENOTSET. The file
server can handle these as it chooses, but the default action is
for it to throw its hands in the air and sing "goodbye, cruel world,
it's over, walk on by".
 1.50  27-Sep-2007  pooka Fix a race in how new cookies are checked. Previously the checking
was done separate of inserting the cookie into the lookup structure
and without any form of interlock. This could lead to the same
cookie pointing to two different nodes. Remedy the race by creating
a separate "checked and ready to be inserted" cookie list which
serves as an interlock without having to hold a fs-global creation
lock.
 1.49  24-Sep-2007  pooka add a few comments and g/c dead code
 1.48  30-Jul-2007  pooka branches: 1.48.4; 1.48.6; 1.48.8; 1.48.10;
Move PUFFS_TYPEPREFIX to puffs_msgif.h since it's used in a macro there.
 1.47  22-Jul-2007  pooka Keep track of the maximum size we have supplied the file server (or
it has supplied us). If we fault pages which are at offset >= server
size, but less than the in-kernel vnode size, inform the file server
of the latest developments in file size before issueing the fault.
The avoids confusion with files which are not written start to finish.

fixes kern/36429 by yamt
 1.46  17-Jul-2007  pooka branches: 1.46.2;
Set a file server supplied file system type in the type field and set
the mntfromname to be the place mounted from instead of the type.
 1.45  01-Jul-2007  pooka Give the file server to ability to request the entire pathname buffer
under lookup by using PUFFS_KFLAG_LOOKUP_FULLPNBUF instead just the
current component.
 1.44  01-Jul-2007  pooka Instead of supplying a plain pid, supply an abstract struct puffs_cid *,
which can currently be used to query the pid and lwpid.
 1.43  01-Jul-2007  pooka make puffs_cred an opaque type
 1.42  24-Jun-2007  pooka Split the NOCACHE option in twain: NOCACHE_NAME & NOCACHE_PAGE.
 1.41  21-Jun-2007  pooka Refactor the pnode2vnode translation slightly so that VFS_ROOT
can use it directly.
 1.40  21-Jun-2007  pooka Reorganize how the root vnode is fetched so that it doesn't always
go through VFS_ROOT() and allow to fetch it without locking it.
This allows us to call the cache flush operations also for the root
vnode and most notably fixes e.g. a "No such file or directory"
for a psshfs root directory ls -l when a file was locally deleted
and remotely re-created.

Also fix some sloppy programming in root node fetch (mostly cosmetic).
 1.39  06-Jun-2007  pooka Move puffs to a two clause license where it already isn't so. And
as agc pointed out, even files with the third clause were already
effectively two clause because of a slight bug in the language...
 1.38  19-May-2007  pooka forgot to commit this with puffs_vnops.c 1.72:

Actually, we do need separate "no references in file server" and
"noref + inactive" flags if we wish to correctly support unix open
file semantics and optimize away pre-reclaim cache flushes. So,
add PNODE_DYING which stands for norefs + inactive.
 1.37  18-May-2007  pooka Introduce noref setbacks, which the file server can use to signal
the kernel it has 0 references to the node in question. In other
words, this can be used to avoid inactive(), or, if the file server
does not implement inactive, prompt reclaim for removed nodes.
 1.36  18-May-2007  pooka Support VOP_POLL. This requires some acrobatics on the puffs_node,
as we give a reference to userspace for the puffs_node for the
duration of the poll call. So reference count puffs_node separately
from the parent vnode. vref()/vrele() is not possible due to a possible
surprise visit from VOP_INACTIVE.
 1.35  17-May-2007  pooka Make it possible for the file server to specify the root vnode type
and other information instead of always using VDIR. To make this
possible without races, require all root node information already
in puffs_mount() and nuke puffs_start2() and the associated start
operation completely.

requested/inspired by Tobias Nygren
 1.34  07-May-2007  pooka Introduce puffs "setbacks", which can be used to set certain flags
for nodes upon return from the userspace. Currently it can be used
to indicate that the file server should be notified of "inactive"
in case the file server has opted to not receive inactive every
time the reference count for a vnode drops to zero. (inactive is
a common event, almost never requires any action and must be executed
sychronously, so it is wasteful).

While doing this, cleanup the release-relock nonsense from the
vntouser*() arguments. It was never enabled and the whole LOCKEDVP()
concept was very broken to begin with.
 1.33  01-May-2007  pooka Fix a problem introduced when I converted puffs to use newlock2:
when unmounting the file system in case of a certain timing (and
possibly some other conditions), a thread would wait on a condition
variable, while another thread broadcast the cv and immediately
proceeded to destroy it. The result was a system frozen completely
solid shorly after the process waiting for the cv woke up. So
introduce reference counting to synchronize destruction of the
resources in unmount.

I was able to repeat the problem only on my laptop in some special
cases, so I do not know how common it was. Ironically, killing
the file server process violently instead of unmount() didn't have
this problem because it never entered the unmount path from two
directions.
 1.32  16-Apr-2007  pooka Give the file server the ability to specify the file handle length
instead of defining a static length file handle on the framework-level.
 1.31  13-Apr-2007  pooka * add fhlen to kernel argument structure
* rename it to puffs_kargs instead of puffs_args
 1.30  04-Apr-2007  pooka Make it possible to interrupt waiters for fs operation completion
again. This is useful until locking is further developed and basically
any deadlocks can be solved by killing appropriate processes.

Thanks especially to Tommi Kyntola and Antti Louko for sitting down
with me and discussing resource ownership and locking strategies
in implementing this.
 1.29  30-Mar-2007  pooka * abstract ASYNCBIOREAD and let callers freely issue a callback called
from putop. even though there's only one user currently, makes code
more readable
* move "delta" to a standard parameter in vntouser and get rid of the
specialcase vntouser_delta
 1.28  29-Mar-2007  pooka Convert spinlocks & sleep/wakeup to newlock2 locking stuff. Fix a
bunch of bugs.

* park structures are now always allocated from a pool instead of a
mixed stack/malloc allocation
* get rid of the whole adjbuf concept, always just alloc the maximal
amount of memory to satisfy a request
* little regression: don't allow interrupting wait from file system
to userspace; this had problems already before, but now the problems
really started to shine through. I'll try to make this work again
some day.
* fix bmap to return a sensible value in runp
 1.27  20-Mar-2007  pooka * rework the page cache interaction a bit: cache metadata in the
kernel and flush it out all at once instead of continuous updating
* add support for delivering notifications to the file server about
when a page was written to (but disabled by default for now). the
file server can use this to request flushing or invalidating the
kernel page cache
 1.26  14-Mar-2007  pooka branches: 1.26.2;
Support B_READ|B_ASYNC in strategy by calling biodone() directly
when the file server puts the result.
 1.25  27-Feb-2007  pooka branches: 1.25.2; 1.25.4;
Make wait for the user file server PCATCHable. This makes it
possible to recover the system by just killing processes in case
a file server manages to recurse into itself either by fault of
file server implementation or by pilot error. The downside is that
the code is extremely hard to follow and practically screams out
for newlock2 (in addition to screaming "bug here"). The whole
PCATCH nonsense and induced megacomplexity can hopefully be avoided
in the future by tweaking other parts of the implementation.
 1.24  15-Feb-2007  pooka branches: 1.24.2;
Hide the debug prints behind PUFFSDEBUG instead of DEBUG. Make the
latter define the former.
 1.23  29-Jan-2007  hubertf Remove more duplicate headers.
Patch by Slava Semushin <slava.semushin@gmail.com>

Again, this was tested by comparing obj files from a pristine and a patched
source tree against an i386/ALL kernel, and also for src/sbin/fsck_ffs,
src/sbin/fsdb and src/usr.sbin/makefs. Only changes in assert() line numbers
were detected in 'objdump -d' output.
 1.22  26-Jan-2007  pooka Initial attempt at suspend/snapshot support for userspace file
servers. This is still pretty much on the level "if it breaks ...".
It should work for single-threaded servers which handle one operation
from start to finish in one go. Also, it does not yet totally
correctly synchronize metadata and data in some cases. So needless
to say, it needs improvement, but it is possible that will have to
wait for some lock revampage.
 1.21  21-Jan-2007  pooka optimize a bit: don't flush pages for vnodes which have no references
in the kernel or links in the backend
 1.20  15-Jan-2007  pooka Store puffs_node's on lists hashed with the cookie value instead
of just one flat list.
 1.19  15-Jan-2007  pooka * do not accept the directory cookie as the result of a lookup (otherwise
we'd be locking against ourselves)
* do not accept duplicate cookies when creating new nodes
 1.18  09-Jan-2007  pooka Introduce flush operations, which the fs server can use to control
kernel caching. Currently supported are only flushing the name
cache for a directory or flushing the name cache for the entire fs.

Also, get rid of PNODE_INACTIVE status, since it was racy and
essentially didn't work. All this on top of being useless in the
first place ....
 1.17  02-Jan-2007  pooka * check userspace version and prevent incompatible mount
* some general maintenance
 1.16  30-Dec-2006  pooka branches: 1.16.2;
* use PUFFS_KFLAG_NOCACHE to also signal that we don't want the namecache
* enter files into the namecache immediately when new nodes are created
(if it's a caching mount, of course)
 1.15  29-Dec-2006  pooka rename the kernel-provided componentname to puffs_kcn; libpuffs now
provides puffs_cn built on top of it
 1.14  10-Dec-2006  pooka Fix a race condition that would cause the mountpoint to be cleaned
from under someone waiting for the fs server response in puffs_unmount()
if the descriptor was closed during the response wait (such as bug
leading to a crash in fs implementation unmount()).
 1.13  05-Dec-2006  pooka branches: 1.13.2;
shuffle functions around a bit: move the transport (/dev/puffs) to
a different file from the messaging (request contents). no functional
change
 1.12  05-Dec-2006  pooka Allow multiple requests to be transferred in each GET/PUTOP. For
a single request, the performance is still the same.
 1.11  01-Dec-2006  pooka prefix kernel flags with PUFFS_KFLAG to have a separate namespace
from the library flags
 1.10  01-Dec-2006  pooka don't call the fs server for all operations, only those it has told
us that it implements
 1.9  18-Nov-2006  pooka branches: 1.9.2;
As a first generation best-effort hack, use NOCACHE to mean "file
size can change without the kernel knowing" and therefore query
the file size before invoking read or write operations.
 1.8  17-Nov-2006  pooka Introduce uncached operation, makes sense when the file system backend
can be modified from elsewhere than the file system interface
 1.7  09-Nov-2006  pooka few renames to better differentiate between mount & start.. plus some
other renaming
 1.6  07-Nov-2006  pooka attach to genfs & support page cache. most noticeable effect is
mmap and therefore execution of binaries starting to work, some
speed improvements with large file I/O also. caching semantics
and error case handling most likely need revisiting.
 1.5  06-Nov-2006  pooka puffs_park always contains a specific puffs_req, so make it a member
instead of a pointer
 1.4  06-Nov-2006  pooka make it possible to build & load puffs as an LKM

by Lubomir Kundrak, PR kern/35000
 1.3  27-Oct-2006  pooka support fifos
 1.2  26-Oct-2006  pooka support specfs
 1.1  22-Oct-2006  pooka kernel portion of puffs - the Pass-to-Userspace Framework File System.
It contains the VFS attachment and userspace message-passing interface.

This work was initially started and completed for Google SoC 2005
and tweaked to work a bit better in the past few weeks. While
being far from complete, it is functional enough to be able and
stable to host a fairly general-purpose in-memory file system in
userspace. Even so, puffs should be considered experimental and
no binary compatibility for interfaces or crash-freedom or zero
security implications should be relied upon just yet.

The GSoC project was mentored by William Studenmund and the final
review for the code was done by Christos.
 1.9.2.4  01-Feb-2007  ad Sync with head.
 1.9.2.3  12-Jan-2007  ad Sync with head.
 1.9.2.2  18-Nov-2006  ad Sync with head.
 1.9.2.1  18-Nov-2006  ad file puffs_sys.h was added on branch newlock2 on 2006-11-18 21:39:20 +0000
 1.13.2.3  18-Dec-2006  yamt sync with head.
 1.13.2.2  10-Dec-2006  yamt sync with head.
 1.13.2.1  05-Dec-2006  yamt file puffs_sys.h was added on branch yamt-splraiseipl on 2006-12-10 07:18:38 +0000
 1.16.2.9  04-Feb-2008  yamt sync with head.
 1.16.2.8  21-Jan-2008  yamt sync with head
 1.16.2.7  07-Dec-2007  yamt sync with head
 1.16.2.6  15-Nov-2007  yamt sync with head.
 1.16.2.5  27-Oct-2007  yamt sync with head.
 1.16.2.4  03-Sep-2007  yamt sync with head.
 1.16.2.3  26-Feb-2007  yamt sync with head.
 1.16.2.2  30-Dec-2006  yamt sync with head.
 1.16.2.1  30-Dec-2006  yamt file puffs_sys.h was added on branch yamt-lazymbuf on 2006-12-30 20:50:01 +0000
 1.24.2.5  17-May-2007  yamt sync with head.
 1.24.2.4  07-May-2007  yamt sync with head.
 1.24.2.3  15-Apr-2007  yamt sync with head.
 1.24.2.2  24-Mar-2007  yamt sync with head.
 1.24.2.1  12-Mar-2007  rmind Sync with HEAD.
 1.25.4.1  11-Jul-2007  mjf Sync with head.
 1.25.2.8  12-Oct-2007  ad Sync with head.
 1.25.2.7  09-Oct-2007  ad Sync with head.
 1.25.2.6  16-Sep-2007  ad Checkpoint work in progress on the vnode lifecycle and reference counting
stuff. This makes it work properly without kernel_lock and fixes a few
quite old bugs. See vfs_subr.c 1.283.2.17 for details.
 1.25.2.5  20-Aug-2007  ad Sync with HEAD.
 1.25.2.4  15-Jul-2007  ad Sync with head.
 1.25.2.3  09-Jun-2007  ad Sync with head.
 1.25.2.2  08-Jun-2007  ad Sync with head.
 1.25.2.1  10-Apr-2007  ad Sync with head.
 1.26.2.1  29-Mar-2007  reinoud Pullup to -current
 1.46.2.1  15-Aug-2007  skrll Sync with HEAD.
 1.48.10.2  30-Jul-2007  pooka Move PUFFS_TYPEPREFIX to puffs_msgif.h since it's used in a macro there.
 1.48.10.1  30-Jul-2007  pooka file puffs_sys.h was added on branch matt-mips64 on 2007-07-30 09:04:59 +0000
 1.48.8.2  14-Oct-2007  yamt sync with head.
 1.48.8.1  06-Oct-2007  yamt sync with head.
 1.48.6.3  23-Mar-2008  matt sync with HEAD
 1.48.6.2  09-Jan-2008  matt sync with HEAD
 1.48.6.1  06-Nov-2007  matt sync with HEAD
 1.48.4.7  09-Dec-2007  jmcneill Sync with HEAD.
 1.48.4.6  21-Nov-2007  joerg Sync with HEAD.
 1.48.4.5  14-Nov-2007  joerg Sync with HEAD.
 1.48.4.4  11-Nov-2007  joerg Sync with HEAD.
 1.48.4.3  26-Oct-2007  joerg Sync with HEAD.

Follow the merge of pmap.c on i386 and amd64 and move
pmap_init_tmp_pgtbl into arch/x86/x86/pmap.c. Modify the ACPI wakeup
code to restore CR4 before jumping back into kernel space as the large
page option might cover that.
 1.48.4.2  07-Oct-2007  joerg Sync with HEAD.
 1.48.4.1  02-Oct-2007  joerg Sync with HEAD.
 1.59.4.4  18-Feb-2008  mjf Sync with HEAD.
 1.59.4.3  27-Dec-2007  mjf Sync with HEAD.
 1.59.4.2  08-Dec-2007  mjf Sync with HEAD.
 1.59.4.1  19-Nov-2007  mjf Sync with HEAD.
 1.59.2.3  21-Nov-2007  bouyer Sync with HEAD
 1.59.2.2  18-Nov-2007  bouyer Sync with HEAD
 1.59.2.1  13-Nov-2007  bouyer Sync with HEAD
 1.65.2.3  26-Dec-2007  ad Sync with head.
 1.65.2.2  08-Dec-2007  ad Sync with head.
 1.65.2.1  04-Dec-2007  ad Pull the vmlocking changes into a new branch.
 1.67.4.2  08-Jan-2008  bouyer Sync with HEAD
 1.67.4.1  02-Jan-2008  bouyer Sync with HEAD
 1.70.28.1  21-Apr-2010  matt sync to netbsd-5
 1.70.20.3  17-Sep-2011  bouyer Pull up following revision(s) (requested by manu in ticket #1666):
sys/fs/puffs/puffs_sys.h: revision 1.78 via patch
sys/fs/puffs/puffs_node.c: revision 1.20 via patch
sys/fs/puffs/puffs_vnops.c: revision 1.155 via patch
Add a mutex for operations that touch size (setattr, getattr, write, fsync).
This is required to avoid data corruption bugs, where a getattr slices
itself within a setattr operation, and sets the size to the stall value
it got from the filesystem. That value is smaller than the one set by
setattr, and the call to uvm_vnp_setsize() trigged a spurious truncate.
The result is a chunk of zeroed data in the file.
Such a situation can easily happen when the ioflush thread issue a
VOP_FSYNC/puffs_vnop_sync/flushvncache/dosetattrn while andother process
do a sys_stat/VOP_GETATTR/puffs_vnop_getattr.
This mutex on size operation can be removed the day we decide VOP_GETATTR
has to operated on a locked vnode, since the other operations that touch
size already require that.
 1.70.20.2  18-Jun-2011  bouyer Pull up following revision(s) (requested by manu in ticket #1623):
lib/libpuffs/puffs.c: revision 1.116
sys/fs/puffs/puffs_vnops.c: revision 1.151
Call advlock method if supplied
 1.70.20.1  09-Jan-2010  snj Pull up following revision(s) (requested by pooka in ticket #1212):
sys/fs/puffs/puffs_msgif.c: revision 1.76 via patch
sys/fs/puffs/puffs_sys.h: revision 1.73 via patch
sys/fs/puffs/puffs_vfsops.c: revision 1.84 via patch
Process flush requests from the file server in a separate thread
context. This fixes a long-standing but seldomly seen deadlock,
where the kernel was holding pages busy (due to e.g. readahead
request) while waiting for the server to respond, and the server
made a callback into the kernel asking to invalidate those pages.
... or, well, theoretically fixes, since I didn't have any reliable
way of repeating the deadlock and I think I saw it only twice.
 1.70.10.2  11-Aug-2010  yamt sync with head.
 1.70.10.1  11-Mar-2010  yamt sync with head
 1.75.4.1  05-Mar-2011  rmind sync with head
 1.75.2.1  17-Aug-2010  uebayasi Sync with HEAD.
 1.78.8.4  27-Feb-2015  martin Pull up following revision(s) (requested by manu in ticket #1260):
lib/libpuffs/puffs.3: revision 1,55,1.60
sys/fs/puffs/puffs_msgif.h: revision 1.84
lib/libperfuse/ops.c: revision 1.83
sys/fs/puffs/puffs_sys.h: revision 1.89
sys/fs/puffs/puffs_vfsops.c: revision 1.116
lib/libperfuse/perfuse.c: revision 1.36
sys/fs/puffs/puffs_vnops.c: revision 1.200-1.202

Use more markup. New sentence, new line. Bump date for previous.

Add PUFFS_KFLAG_NOFLUSH_META to prevent sending metadata flush to FUSE
FUSE filesystems do not expect to get metadata updates for [amc]time
and size, they updates the value on their own after operations.

The PUFFS PUFFS_KFLAG_NOFLUSH_META option prevents regular metadata cache
flushes to the filesystem , and libperfuse uses it to match Linux FUSE
behavior.

While there, fix a bug in SETATTR: do not update kernel metadata cache
from SETATTR reply when the request is asynchronous, as we do not have
the reply yet.

Update file size after write without metadata flush
If we do not use metadata flush, we must make sure the size is updated
in the filesystem after a write, otherwise the next GETATTR will get us
a stale value and the file will be truncated.
 1.78.8.3  03-Nov-2014  msaitoh Pull up following revision(s) (requested by manu in ticket #1140):
lib/libperfuse/ops.c 1.63-1.69
lib/libperfuse/perfuse.c 1.32-1.33
lib/libperfuse/perfuse_priv.h 1.32-1.34
lib/libperfuse/subr.c 1.20
lib/libpuffs/creds.c 1.16
lib/libpuffs/dispatcher.c 1.47
lib/libpuffs/puffs.h 1.125
lib/libpuffs/puffs_ops.3 1.37-1.38
lib/libpuffs/requests.c 1.24
sys/fs/puffs/puffs_msgif.h 1.81
sys/fs/puffs/puffs_sys.h 1.85
sys/fs/puffs/puffs_vnops.c 1.183
usr.sbin/perfused/msg.c 1.22
Bring libpuffs, libperfuse and perfused on par with -current:
- implement FUSE direct I/O
- remove useless code and warnings
- fix missing GETATTR bugs
- fix exended attribute get and list operations
 1.78.8.2  12-Aug-2012  martin Pull up following revision(s) (requested by manu in ticket #438):
lib/libperfuse/perfuse_priv.h: revision 1.31
sys/fs/puffs/puffs_msgif.h: revision 1.80
sys/fs/puffs/puffs_vnops.c: revision 1.171
lib/libpuffs/puffs_ops.3: revision 1.31
sys/fs/puffs/puffs_vnops.c: revision 1.172
sys/fs/puffs/puffs_vnops.c: revision 1.173
sys/fs/puffs/puffs_vnops.c: revision 1.174
usr.sbin/perfused/perfused.c: revision 1.24
sys/fs/puffs/puffs_sys.h: revision 1.80
sys/fs/puffs/puffs_sys.h: revision 1.81
sys/fs/puffs/puffs_sys.h: revision 1.82
lib/libperfuse/subr.c: revision 1.19
lib/libperfuse/perfuse.c: revision 1.30
sys/fs/puffs/puffs_msgif.c: revision 1.90
sys/fs/puffs/puffs_msgif.c: revision 1.91
sys/fs/puffs/puffs_msgif.c: revision 1.92
lib/libperfuse/ops.c: revision 1.59
lib/libpuffs/puffs.3: revision 1.53
lib/libperfuse/debug.c: revision 1.12
lib/libpuffs/puffs.3: revision 1.54
sys/fs/puffs/puffs_vnops.c: revision 1.167
sys/fs/puffs/puffs_msgif.h: revision 1.79
usr.sbin/perfused/msg.c: revision 1.21
sys/fs/puffs/puffs_vfsops.c: revision 1.102
sys/fs/puffs/puffs_vfsops.c: revision 1.103
sys/fs/puffs/puffs_vfsops.c: revision 1.105
lib/libpuffs/puffs.h: revision 1.123
lib/libperfuse/perfuse_if.h: revision 1.20
lib/libperfuse/perfuse.c: revision 1.29
lib/libpuffs/dispatcher.c: revision 1.42
lib/libpuffs/dispatcher.c: revision 1.43
- Fix same vnodes associated with multiple cookies
The scheme used to retreive known nodes on lookup was flawed, as it only
used parent and name. This produced a different cookie for the same file
if it was renamed, when looking up ../ or when dealing with multiple files
associated with the same name through link(2).
We therefore abandon the use of node name and introduce hashed lists of
inodes. This causes a huge rewrite of reclaim code, which do not attempt
to keep parents allocated until all their children are reclaimed
- Fix race conditions in reclaim
There are a few situations where we issue multiple FUSE operations for
a PUFFS operation. On reclaim, we therefore have to wait for all FUSE
operation to complete, not just the current exchanges. We do this by
introducing node reference count with node_ref() and node_rele().
- Detect data loss caused by FAF
VOP_PUTPAGES causes FAF writes where the kernel does not check the
operation result. At least issue a warning on error.
- Enjoy FAF shortcut on setattr
No need to wait for the result if the kernel does not want it. There is
however an exception for setattr that touch the size, we need to wait
for completion because we have other operations queued for after the
resize.
- Fix fchmod() on write-open file
fchmod() on a node open with write privilege will send setattr with both mode
and size set. This confuses some FUSE filesystem. Therefore we send two FUSE
operations, one for mode, and one for size.
- Remove node TTL handling for netbsd-5 for simplicity sake. The code
still builds on netbsd-5 but does not have the node TTL feature anymore.
It works fine with kernel support on netbsd-6.
- Improve PUFFS_KFLAG_CACHE_FS_TTL by reclaiming older inactive nodes.
The normal kernel behavior is to retain inactive nodes in the freelist
until it runs out of vnodes. This has some merit for local filesystems,
where the cost of an allocation is about the same as the cost of a
lookup. But that situation is not true for distributed filesystems.
On the other hand, keeping inactive nodes for a long time hold memory
in the file server process, and when the kernel runs out of vnodes, it
produce reclaim avalanches that increase lattency for other operations.
We do not reclaim inactive vnodes immediatly either, as they may be
looked up again shortly. Instead we introduce a grace time and we
reclaim nodes that have been inactive beyond the grace time.
- Fix lookup/reclaim race condition.
The above improvement undercovered a race condition between lookup and
reclaim. If we reclaimed a vnode associated with a userland cookie while
a lookup returning that same cookiewas inprogress, then the kernel ends
up with a vnode associated with a cookie that has been reclaimed in
userland. Next operation on the cookie will crash (or at least confuse)
the filesystem.
We fix this by introducing a lookup count in kernel and userland. On
reclaim, the kernel sends the count, which enable userland to detect
situation where it initiated a lookup that is not completed in kernel.
In such a situation, the reclaim must be ignored, as the node is about
to be looked up again.
Fix hang unmount bug introduced by last commit.
We introduced a slow queue for delayed reclaims, while the existing
queue for unmount, flush and exist has been renamed fast queue. Both
queues had timestamp for when an operation should be done, but it was
useless for the fast queue, which is always used to run an operation
ASAP. And the timestamp test had an error that turned ASAP into "at next
tick", but nobody what there to wake the thread at next tick, hence
the hang. The fix is to remove the useless and buggy timestamp test for
fast queue.
Rename slow sopreq queue into node sopreq queue, to refet the fact that
is only intended for postponed node reclaims.
When purging the node sopreq queue, do not call puffs_msg_sendresp(), as
it makes no sense.
Fix race condition between (create|mknod|mkdir|symlino) and reclaim, just
like we did it between lookup and reclaim.
Missing bit in previous commit (prevent race between create|mknod|mkdir|symlink
and reclaim)
Bump date for previous.
New sentence, new line; remove trailing whitespace; fix typos;
punctuation nits.
Add PUFFS_KFLAG_CACHE_DOTDOT so that vnodes hold a reference on their
parent, keeping them active, and allowing to lookup .. without sending
a request to the filesystem.
Enable the featuure for perfused, as this is how FUSE works.
Missing bit in previous commit (PUFFS_KFLAG_CACHE_DOTDOT option to avoid
looking up ..)
 1.78.8.1  23-Apr-2012  riz Pull up following revision(s) (requested by manu in ticket #195):
lib/libskey/skeysubr.c: revision 1.27
lib/libkvm/kvm_getloadavg.c: revision 1.11
lib/libwrap/update.c: revision 1.9
lib/liby/yyerror.c: revision 1.9
lib/libpuffs/puffs_ops.3: revision 1.30
lib/libwrap/misc.c: revision 1.10
lib/libwrap/hosts_access.c: revision 1.20
lib/libpuffs/pnode.c: revision 1.11
lib/libperfuse/subr.c: revision 1.17
lib/libpuffs/pnode.c: revision 1.12
lib/libperfuse/subr.c: revision 1.18
lib/libwrap/options.c: revision 1.15
lib/libwrap/fix_options.c: revision 1.11
lib/libperfuse/ops.c: revision 1.52
lib/libperfuse/ops.c: revision 1.53
lib/libperfuse/ops.c: revision 1.54
lib/libwrap/hosts_ctl.c: revision 1.5
lib/libintl/gettext.c: revision 1.27
lib/libwrap/shell_cmd.c: revision 1.6
lib/libpuffs/dispatcher.c: revision 1.39
lib/libperfuse/perfuse_priv.h: revision 1.27
lib/libwrap/socket.c: revision 1.19
lib/libpuffs/puffs.3: revision 1.50
lib/libperfuse/perfuse_priv.h: revision 1.28
lib/libpuffs/puffs_priv.h: revision 1.45
lib/libpuffs/puffs.3: revision 1.51
lib/libperfuse/perfuse_priv.h: revision 1.29
lib/libwrap/percent_x.c: revision 1.5
lib/libpuffs/puffs.3: revision 1.52
lib/libperfuse/debug.c: revision 1.11
sys/fs/puffs/puffs_vnops.c: revision 1.165
lib/libwrap/tcpd.h: revision 1.13
sys/fs/puffs/puffs_vnops.c: revision 1.166
lib/libwrap/eval.c: revision 1.7
sys/fs/puffs/puffs_msgif.h: revision 1.78
sys/fs/puffs/puffs_vfsops.c: revision 1.101
lib/libwrap/rfc931.c: revision 1.9
lib/libwrap/clean_exit.c: revision 1.5
lib/libpuffs/puffs.h: revision 1.120
lib/libc/stdlib/jemalloc.c: revision 1.27
lib/librmt/rmtlib.c: revision 1.26
lib/libpuffs/puffs.h: revision 1.121
sys/fs/puffs/puffs_sys.h: revision 1.79
lib/librumpclient/rumpclient.c: revision 1.48
lib/libwrap/refuse.c: revision 1.5
lib/libperfuse/perfuse.c: revision 1.26
lib/libperfuse/perfuse.c: revision 1.27
tests/fs/puffs/t_fuzz.c: revision 1.5
lib/libperfuse/perfuse.c: revision 1.28
lib/libpuffs/dispatcher.c: revision 1.40
sys/fs/puffs/puffs_node.c: revision 1.24
lib/libwrap/diag.c: revision 1.9
lib/libintl/textdomain.c: revision 1.13
Use C89 function definition
Add name and atttribute cache with filesytem provided TTL.
lookup, create, mknod, mkdir, symlink, getattr and setattr messages
have been extended so that attributes and their TTL can be provided
by the filesytem. lookup, create, mknod, mkdir, and symlink messages
are also extended so that the filesystem can provide name TTL.
Add PUFFS_KFLAG_CACHE_FS_TTL flag to puffs_init(3) to use name and
attribute cache with filesystem provided TTL.
lookup, create, mknod, mkdir, symlink, getattr and setattr messages
have been extended so that attributes and their TTL can be provided
by the filesytem. lookup, create, mknod, mkdir, and symlink messages
are also extended so that the filesystem can provide name TTL.
The filesystem updates attributes and TTL using
puffs_pn_getvap(3), puffs_pn_getvattl(3), and puffs_pn_getcnttl(3)
Use new PUFFS_KFLAG_CACHE_FS_TTL option to puffs_init(3) so that
FUSE TTL on name and attributes are used. This save many PUFFS
operations and improves performances.
PUFFS_KFLAG_CACHE_FS_TTL is #ifdef'ed in many places for now so that
libperfuse can still be used on netbsd-5.
Split file system.
Comma fixes.
Remove dangling &quot;and&quot;.
Bump date for previous.
- Makesure update_va does not change vnode size when it should not. For
instance when doing a fault-issued VOP_GETPAGES within VOP_WRITE, changing
size leads to panic: genfs_getpages: past eof.
-Handle ticks wrap around for vnode name andattribute timeout
- When using PUFFS_KFLAG_CACHE_FS_TTL, do not use puffs_node to carry
attribute and TTL fora newly created node. Instead extend puffs_newinfo
and add puffs_newinfo_setva() and puffs_newinfo_setttl()
- Remove node_mk_common_final in libperfuse. It used to set uid/gid for
a newly created vnode but has been made redundant along time ago since
uid and gid are properly set in FUSE header.
- In libperfuse, check for corner case where opc = 0 on INACTIVE and RECLAIM
(how is it possible? Check for it to avoid a crash anyway)
- In libperfuse, make sure we unlimit RLIMIT_AS and RLIMIT_DATA so that
we do notrun out of memory because the kernel is lazy at reclaiming vnodes.
- In libperfuse, cleanup style of perfuse_destroy_pn()
Do not set PUFFS_KFLAG_CACHE_FS_TTL for PUFFS tests
 1.78.6.1  29-Apr-2012  mrg sync to latest -current.
 1.78.2.3  22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.78.2.2  30-Oct-2012  yamt sync with head
 1.78.2.1  17-Apr-2012  yamt sync with head
 1.82.2.3  03-Dec-2017  jdolecek update from HEAD
 1.82.2.2  20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.82.2.1  23-Jun-2013  tls resync from head
 1.83.6.1  18-May-2014  rmind sync with head
 1.84.4.3  27-Feb-2015  martin Pull up following revision(s) (requested by manu in ticket #555):
lib/libpuffs/puffs.3: revision 1.60
sys/fs/puffs/puffs_msgif.h: revision 1.84
lib/libperfuse/ops.c: revision 1.83
sys/fs/puffs/puffs_sys.h: revision 1.89
sys/fs/puffs/puffs_vfsops.c: revision 1.116
lib/libperfuse/perfuse.c: revision 1.36
sys/fs/puffs/puffs_vnops.c: revision 1.200-1.202

Add PUFFS_KFLAG_NOFLUSH_META to prevent sending metadata flush to FUSE

FUSE filesystems do not expect to get metadata updates for [amc]time
and size, they updates the value on their own after operations.

The PUFFS PUFFS_KFLAG_NOFLUSH_META option prevents regular metadata cache
flushes to the filesystem , and libperfuse uses it to match Linux FUSE
behavior.

While there, fix a bug in SETATTR: do not update kernel metadata cache
from SETATTR reply when the request is asynchronous, as we do not have
the reply yet.

Update file size after write without metadata flush
If we do not use metadata flush, we must make sure the size is updated
in the filesystem after a write, otherwise the next GETATTR will get us
a stale value and the file will be truncated.
 1.84.4.2  29-Aug-2014  martin Pull up following revision(s) (requested by hannken in ticket #67):
sys/fs/puffs/puffs_sys.h: revision 1.86
sys/fs/puffs/puffs_vfsops.c: revision 1.114
sys/fs/puffs/puffs_msgif.c: revision 1.95
sys/fs/puffs/puffs_node.c: revision 1.32
sys/fs/puffs/puffs_vnops.c: revision 1.184
Change puffs from hashlist to vcache.
- field "pa_nhashbuckets" of struct "puffs_kargs" becomes a no-op.
and should be removed on the next protocol version bump.
 1.84.4.1  26-Aug-2014  riz Pull up following revision(s) (requested by manu in ticket #52):
sys/fs/puffs/puffs_msgif.h: revision 1.81
sys/fs/puffs/puffs_sys.h: revision 1.85
sys/fs/puffs/puffs_vnops.c: revision 1.183
Add a oflags input field to open requests so that the filesystem can pass
back information about the file. Implement PUFFS_OPEN_IO_DIRECT, which
will force direct IO (bypassing page cache) for the file.
 1.88.2.2  09-Jul-2016  skrll Sync with HEAD
 1.88.2.1  06-Apr-2015  skrll Sync with HEAD
 1.90.18.1  10-Jun-2019  christos Sync with HEAD
 1.90.16.3  14-Jan-2019  pgoyette Create a variant of the HOOK macros that handles hook routines of
type void, and use them where appropriate.
 1.90.16.2  17-Sep-2018  pgoyette Adapt (most of) the indirect function pointers to the new MP-safe
mechanism. Still remaining are the compat_netbsd32 stuff, and
some usb subroutines.
 1.90.16.1  24-Mar-2018  pgoyette Add fs/puffs compat_50 to the modules

RSS XML Feed