Home | History | Annotate | Download | only in libpuffs
History log of /src/lib/libpuffs/puffs.h
RevisionDateAuthorComments
 1.130  03-Dec-2021  pho Use size_t in place of voff_t in puffs_setrootinfo(3)

Chuck Silvers pointed out that voff_t was also supposed to be
kernel-only. The correct type to use in userland would be off_t, but
since changing vsize_t to either voff_t or off_t is an ABI change on
32-bit platforms, we use size_t knowing that it is technically
incorrect.
 1.129  03-Dec-2021  pho Avoid using register_t or vsize_t in <puffs.h>

<puffs.h> is a user-space header, and should not use any of
kernel-only types. It's not reasonable to require user-land
filesystems to #define _KERNTYPES.
 1.128  23-Sep-2019  christos Restore binary compatibility by using the statvfs90 structure internally.
 1.127  12-Nov-2015  christos branches: 1.127.16;
consistency fixes.
 1.126  31-Oct-2014  manu libpuffs support for fallocate and fdiscard operations
 1.125  16-Aug-2014  manu Add an open2 method, like open but with an additionnal oflags used
by the filesystem to send back information about the file.
This is used to implement PUFFS_OPEN_IO_DIRECT by which the filesystem
tells the kernel that read/write should bypass the page cache.
 1.124  16-Aug-2012  manu branches: 1.124.10;
Fix regression that has been introduced when the lookup/reclaim race
condition was addressed in libpuffs by counting lookups.

The fix assumes that cookies map to struct puffs_cookie, which has not
been documented as a requirement for filesystems using libpuffs. As an
example, we got burnt by this assumption in libp2k (kern/46734), and
we fixed bit by actually mapping libp2k cookies to struct puffs_node.

It is unlikely, but there may be third party filesystems that use cookies
unmapped to struct puffs_node, and they were left broken for now.

- we introduce a puffs_init() flag PUFFS_FLAG_PNCOOKIE that let filesystems
inform libpuffs that they map cookies to struct puffs_node. Is that flag
is used, the lookup/reclaim race condition fix is enabled. We enable the
flag for libp2k.

- filesystems that use puffs_pn_new() obviouslty use struct puffs_node
and gain PUFFS_FLAG_PNCOOKIE automatically even if they did not specify
it in puffs_init(). This include all our PUFFS filesystem in-tree except
libp2k.

- for filesystems not willing to use struct puffs_node, we introduce a
reclaim2 vnop, which is reclaim with an additionnal lookup count argument.
This vnop let the filesystem implement the lookup/reclaim race fix on
its own.
 1.123  21-Jul-2012  manu - Improve PUFFS_KFLAG_CACHE_FS_TTL by reclaiming older inactive nodes.

The normal kernel behavior is to retain inactive nodes in the freelist
until it runs out of vnodes. This has some merit for local filesystems,
where the cost of an allocation is about the same as the cost of a
lookup. But that situation is not true for distributed filesystems.
On the other hand, keeping inactive nodes for a long time hold memory
in the file server process, and when the kernel runs out of vnodes, it
produce reclaim avalanches that increase lattency for other operations.

We do not reclaim inactive vnodes immediatly either, as they may be
looked up again shortly. Instead we introduce a grace time and we
reclaim nodes that have been inactive beyond the grace time.

- Fix lookup/reclaim race condition.

The above improvement undercovered a race condition between lookup and
reclaim. If we reclaimed a vnode associated with a userland cookie while
a lookup returning that same cookiewas inprogress, then the kernel ends
up with a vnode associated with a cookie that has been reclaimed in
userland. Next operation on the cookie will crash (or at least confuse)
the filesystem.

We fix this by introducing a lookup count in kernel and userland. On
reclaim, the kernel sends the count, which enable userland to detect
situation where it initiated a lookup that is not completed in kernel.
In such a situation, the reclaim must be ignored, as the node is about
to be looked up again.
 1.122  27-Jun-2012  manu Pass the FAF flag to handlers:
- setattr_ttl is updated to add a flag argument. Since it was not present in
a previous release, we can change its API
- write2 is introduced, this is write with an extra flag for FAF.
- fsync already has the FAF information in a flag and needs no change
- for other operations, FAF is unconditional
 1.121  18-Apr-2012  manu - When using PUFFS_KFLAG_CACHE_FS_TTL, do not use puffs_node to carry
attribute and TTL fora newly created node. Instead extend puffs_newinfo
and add puffs_newinfo_setva() and puffs_newinfo_setttl()
- Remove node_mk_common_final in libperfuse. It used to set uid/gid for
a newly created vnode but has been made redundant along time ago since
uid and gid are properly set in FUSE header.
- In libperfuse, check for corner case where opc = 0 on INACTIVE and RECLAIM (how is it possible? Check for it to avoid a crash anyway)
- In libperfuse, make sure we unlimit RLIMIT_AS and RLIMIT_DATA so that
we do notrun out of memory because the kernel is lazy at reclaiming vnodes.
- In libperfuse, cleanup style of perfuse_destroy_pn()
 1.120  08-Apr-2012  manu Add PUFFS_KFLAG_CACHE_FS_TTL flag to puffs_init(3) to use name and
attribute cache with filesystem provided TTL.
lookup, create, mknod, mkdir, symlink, getattr and setattr messages
have been extended so that attributes and their TTL can be provided
by the filesytem. lookup, create, mknod, mkdir, and symlink messages
are also extended so that the filesystem can provide name TTL.
The filesystem updates attributes and TTL using
puffs_pn_getvap(3), puffs_pn_getvattl(3), and puffs_pn_getcnttl(3)
 1.119  29-Aug-2011  joerg branches: 1.119.2; 1.119.4;
Use __dead
 1.118  04-Jul-2011  manu Add a flag to VOP_LISTEXTATTR(9) so that the vnode interface can tell the
filesystem in which format extended attribute shall be listed.

There are currently two formats:
- NUL-terminated strings, used for listxattr(2), this is the default.
- one byte length-pprefixed, non NUL-terminated strings, used for
extattr_list_file(2), which is obtanined by setting the
EXTATTR_LIST_PREFIXLEN flag to VOP_LISTEXTATTR(9)

This approach avoid the need for converting the list back and forth, except
in libperfuse, since FUSE uses NUL-terminated strings, and the kernel may
have requested EXTATTR_LIST_PREFIXLEN.
 1.117  07-Jun-2010  pooka Make retval argument for pathconf a register_t to match VOP_PATHCONF.
This makes the size the same on 64bit archs. Don't bother bumping
any version, since you'd have explicitly had to jump through some
hoops to use pathconf before.
 1.116  21-May-2010  pooka add option string for no attribute cache
(foreseeing the odd event I might actually implement one some day)
 1.115  21-May-2010  pooka * support extended attributes
* bump major due to structure growth
* add some spare space
* remove ABI sillyness
 1.114  12-Jan-2010  pooka Add convenience routine puffs_unmountonsignal(), which does exactly that.
 1.113  05-Dec-2009  pooka Add an error handler which syslogs but does not abort() and make it
the default. Most servers just shovel bits from their backend to
the kernel without checking the contents and dumping core in case
the backend gives garbage is a bit harsh.
 1.112  18-Oct-2009  pooka Make it possible to supply parameters to puffs_init() later (but
before puffs_mount()).
 1.111  17-Oct-2009  pooka Dispatch puffs_node_abortop().

Note: We use the storage of puffs_cache_write from puffs_ops for
this purpose. It's not issued by the kernel and hence currently
unused, and this saves us from the trouble of bumping the lib major
version.
 1.110  12-Dec-2008  pooka Add puffs_cancel(), which allows to back down after puffs_init()
and before puffs_mount().
(this should actually be integrated with puffs_exit(), but that
routine is currently so non-sensible that I didn't bother now)
 1.109  12-Dec-2008  pooka add:
puffs_setspecific(pu, private)
Can be used to set the specific data after the call to
puffs_init().
 1.108  12-Aug-2008  pooka branches: 1.108.4; 1.108.8;
Replace void * by puffs_cookie_t where appropriate. No functional change.
 1.107  29-Jan-2008  pooka branches: 1.107.6;
Fix a weirdosity in the framebuf gotfb interface: hand buffer
ownership to the called routine instead of keeping it. This means
that callees must now destroy the buffer once they are done with it.
 1.106  28-Jan-2008  pooka Bump version for lazy contexts (probably not necessary, but it's
been a long time since I bumbed the version ;)
 1.105  28-Jan-2008  pooka Implement lazy contexts for file systems using puffs_mainloop().
Previously each request was executed on its own callcontext and
switched to every time the request was being processed. Now requests
borrow the mainloop context and switch only if/when they yield.
This takes two context switches away from a file system request
bringing down the typical amounts 2->0 (e.g. dtfs) and 4->2 (e.g.
psshfs).

The interfaces for manually executing requests changed a bit:
puffs_dispatch_create() and puffs_dispatch_exec() must now be used.
They are not tested, as nothing in-tree wants them and I doubt
anyone else is really interested in them either.

Also do some misc code cleanup related to execution contexts. The
"work-in-progress checkpoint" committed over a year ago was starting
to look slightly weed-infested.
 1.104  14-Jan-2008  pooka Introduce PUFFS_STACKSIZE_MIN to avoid overloading 0 and getting
a warning from the library when really wanting to say "just the
minimum, please".
 1.103  25-Dec-2007  pooka Add a public interface for setting a cookie mapper.
 1.102  19-Dec-2007  pooka Add generic getattr routine.
 1.101  16-Dec-2007  pooka * nuke puffs_cc_get{specific,usermount} for good
* move prototypes for puffs_docc and puffs_dopufbuf into the
public header, as they are should be exposed
 1.100  11-Dec-2007  pooka nuke #define whose purpose has ceased to be
 1.99  08-Dec-2007  pooka Now that "l" is gone both as an argument to operations and from
componentname, remove all vestiges of puffs_cid.
 1.98  04-Dec-2007  pooka Part 3/n of extensive changes to message handling:

Get rid of the original puffs_req(3) framework and use puffs_framebuf(3)
instead for file system requests. It has the advantage of being
suitable for transporting a distributed message passing protocol
and therefore us being able to run the file system server on any
host.

Ok, puffs is not quite here yet: libpuffs needs to grow request
routing support and the message contents need to be munged into a
host independent format. Saying which format would be telling,
but it might begin with an X, end in an L and have the 13th character
in the middle. Keep an eye out for the sequels: Parts 3+m/n.
 1.97  30-Nov-2007  pooka Rototill.

Ok, ok, a few more words about it: stop holding puffs_cc as a holy
value and passing it around to almost every possible place (popquiz:
which kernel variable does this remind you of?). Instead, pass
the natural choice, puffs_usermount, and fetch puffs_cc via
puffs_cc_getcc() only in routines which actually need it. This
not only simplifies code, but (thanks to the introduction of
puffs_cc_getcc()) enables constructs which weren't previously sanely
possible, say layering as a curious example.

There's still a little to do on this front, but this was the major
fs interface blast.
 1.96  29-Nov-2007  pooka * allocate pcc as a slice from the stack instead of allocating it
separately
* provide puffs_cc_getcc()

This is in preparation for the removal of you-should-guess-what as
an argument to routines here and there and everywhere.
 1.95  27-Nov-2007  pooka Remove "puffs_cid" from the puffs interface following l-removal
from the kernel vfs interfaces. puffs_cc_getcaller(pcc) can be
used now should the same information be desired.
 1.94  16-Nov-2007  pooka Introduce puffs_daemon(), which works like daemon(3) except that it
also synchronizes with puffs_mount() and does not return (exit) in the
parent process until the file system has been mounted. This makes
it possible to reliably run e.g. mount_foo jippi /kai ; cd /kai/ee
 1.93  05-Nov-2007  pooka remember to bump version for puffs_mainloop() change
 1.92  05-Nov-2007  pooka Pull the daemonizing code out of the library mainloop into the file
servers. Calling daemon() (i.e. fork()ing) inside a library can
cause nice surprises for e.g. threaded programs. As discussed with
Greg Oster & others.
 1.91  28-Oct-2007  pooka Add possibility to set pre- and post callbacks which will be called
for all operations before or after executing the specific callback
(the api is not be final yet, though).
 1.90  26-Oct-2007  pooka mmap stacks, makes it possible to align them
 1.89  21-Oct-2007  pooka Add puffs_cc_schedule() which marks a pcc runnable and will pass
execution to it when in the main loop the next time.
 1.88  21-Oct-2007  pooka Always provide caller information from the kernel based on curlwp.
(but don't deprecate the old puffs_cid interface just yet)
 1.87  18-Oct-2007  pooka * Change type of boolean functions to, well, boolean instead of int.
* add puffs_cid_isequal() which can be used to test if the two caller
id's refer to the same process/lwp.
 1.86  11-Oct-2007  pooka Part 1/n of some pretty extensive changes to how the kernel module
interacts with the userspace file server:

* since the kernel-user communication is not purely request-response
anymore (hasn't been since 2006), try to rename some "request" to
"message". more similar mangling will take place in the future.

* completely rework how messages are allocated. previously most of
them were borrowed from the stack (originally *all* of them),
but now always allocate dynamically. this makes the structure
of the code much cleaner. also makes it possible to fix a
locking order violation. it enables plenty of future enhancements.

* start generalizing the transport interface to be independent of puffs

* move transport interface to read/write instead of ioctl. the
old one had legacy design problems, and besides, ioctl's suck.
implement a very generic version for now; this will be
worked on later hopefully some day reaching "highly optimized".

* implement libpuffs support behind existing library request
interfaces. this will change eventually (I hate those interfaces)
 1.85  01-Oct-2007  pooka Pass reason string to pu_errnotify. Print it in the default
version before aborting.
 1.84  27-Sep-2007  pooka Add error notifications, which are used to deliver errors from the
kernel to the file server for silly things the file server did,
e.g. attempting to create a file with size VSIZENOTSET. The file
server can handle these as it chooses, but the default action is
for it to throw its hands in the air and sing "goodbye, cruel world,
it's over, walk on by".
 1.83  06-Sep-2007  pooka Give cmpframe() the ability to shortcircuit the comparison loop by
signalling back that the buffer under examination is not a response
at all.
 1.82  01-Sep-2007  pooka Add gotframe callback, which is called if an incoming frame matches
no outstanding request.
 1.81  22-Aug-2007  pooka branches: 1.81.2;
Mimic namei structure changes for puffs. bump both kernel & lib version.
 1.80  18-Aug-2007  pooka puffs_cc_create/destroy aren't supposed to be exported to the user
 1.79  17-Aug-2007  pooka Add accessors for struct puffs_node to prepare for removal from
exported headers.
 1.78  15-Aug-2007  pooka Nuke PUFFSLOOKUP_FOO and move to NAMEI_FOO
 1.77  27-Jul-2007  pooka branches: 1.77.4;
mmap fflags -> prot
 1.76  20-Jul-2007  pooka Add puffs_framev_enqueue_waitevent, which waits for read/write/error
events from an i/o descriptor. It yields until an event happens
and continues execution when the event loop notices the event has
happened.
 1.75  17-Jul-2007  pooka * add mntfromname parameter to puffs_init()
* set it in file servers
 1.74  09-Jul-2007  pooka add IO_NDELAY
 1.73  08-Jul-2007  pooka g/c leftover
 1.72  08-Jul-2007  pooka make notifyfunction optional and do not call removeonclose() if
none is given.
 1.71  07-Jul-2007  pooka Make puffs_framebuf better suited from environments with multiple sources:

* in addition add/remove, allow enable/disable, which can be used
to control events for descriptors without having to remove all the
data associated with them
* add directsend/receive, which can be used to pass the same buffer
from the caller to read/writeframe and back again
* add flags to enqueue functions and allow urgent buffers to be
processed as the next PDU
 1.70  02-Jul-2007  pooka Get rid of the "int *refs" parameter to inactive: the same can be
accomplished now with puffs_setbacks.
 1.69  01-Jul-2007  pooka Instead of requesting various pieces of information from node-creating
methods using "please fill me"-pointers, pass "struct puffs_newinfo *"
and provide methods to fill in new node info.
 1.68  01-Jul-2007  pooka Instead of supplying a plain pid, supply an abstract struct puffs_cid *,
which can currently be used to query the pid and lwpid.
 1.67  01-Jul-2007  pooka make puffs_cred an opaque type
 1.66  24-Jun-2007  pooka Actually, keep PUFFS_KFLAG_NOCACHE and -o cache around as shorthand
to neither page- nor namecache.
 1.65  24-Jun-2007  pooka PUFFS_KFLAG_NOCACHE became two, so introduce the command line options
-o nonamecache and -o nopagecache and adapt file systems where necessary.
 1.64  24-Jun-2007  pooka Introduce puffs_null_setops(), which initializes the ops vector with
puffs nullfs ops.
 1.63  24-Jun-2007  pooka Introduce puffs_pn_remove() to signal that a node has been deleted
and destroyed, but not yet reclaimed. This prevents puffs_pn_nodewalk()
from returning stale entries. Make nullfs use this (some file
systems are a bit too happy with recycling inode numbers).
 1.62  06-Jun-2007  pooka Move puffs to a two clause license where it already isn't so. And
as agc pointed out, even files with the third clause were already
effectively two clause because of a slight bug in the language...
 1.61  20-May-2007  pooka Introduce puffs_framev_framebuf_ccpromote, which can be used to
turn a reference to puffs_framebuf in the file system from a
cb/justsend operation to a cc wait, should the file system find
itself desiring the result.
 1.60  20-May-2007  pooka Change the signature of puffs_respcmp to return -1,0,1 for smaller,
equal, larger, respectively instead of 0/1 for non/equal. This
will allow sorting the buffers for faster matching in libpuffs.
While here, change the name from respcmp to framecmp, as that better
reflects the purpose.

NOTE! there is no obvious way to make compilation fail for file
systems which may already be using this feature (although I don't
think there are any outside our tree, as the feature is two weeks
old). Nevertheless, non-updated file systems will fail very quickly.
 1.59  18-May-2007  pooka dispatch PUFFS_VN_POLL
 1.58  17-May-2007  pooka Reflect kernel changes for supplying root node information in mount
instead of puffs_start(). Get completely rid of puffs_start(), as
everything it used to do is now handled by the mount routine.
Introduce an optional pre-mount call puffs_setrootinfo() for setting
non-default root node information. As the old puffs_mount() is
now virtually useless, say byebye to it and rename the old
puffs_domount() to puffs_mount(), but add a root cookie parameter
to compensate for the late puffs_start().
 1.57  16-May-2007  pooka Add error parameter to puffs_framev_cb to signal if there is a
result of if the function should just do resource cleanup.
 1.56  15-May-2007  pooka Step 3 and currently final step of a more general purpose event loop:
support removal and addition of i/o file descriptors on the fly.

* detect closed file descriptors
* automatically free waiters of a dead file descriptor
* give the file server the possibility to specify a callback which
notifies of a dead file descriptor
* move loop function to be a property of the mainloop instead of
framebuf (doesn't change effective behaviour)
* add the possibility to configure a timespec parameter which
attempts to call the loop function periodically
* move the event loop functions from the puffs_framebuf namespace
to puffs_framev to differential between pure memory management
functions
 1.55  11-May-2007  pooka Step 2 of general-purpose eventloop: get rid of puffs_framebuf_eventloop().
puffs_mainloop() now handles that functionality as well.
 1.54  11-May-2007  pooka Step 1 in moving towards a general-purpose eventloop: allow the
framebuf event loop to take n i/o fd's as parameters and also allow
dynamic add/remove of fd's. (not tested except for one fd, but more
changes coming soon)
 1.53  09-May-2007  pooka Remove the puffs_usermount argument from puffs_req_handle(). It can
be found from the other arguments and this way puffs_req_handle() is
more symmetric with the other puffs_req() routines.
 1.52  07-May-2007  pooka Add puffs_setback(), which can be used to set setbacks for nodes before
the call returns to the kernel. Currently it can be called only from
open(), mmap(), remove() and rmdir(). (But that might change)
 1.51  06-May-2007  pooka Remove the user-specified argument for the framebuf loop callback.
It can be generated through other means if it is desired.
 1.50  05-May-2007  pooka Add puffs "frame buffers", which are an abstraction of the buffering
and event handling mechanisms required in file servers with blocking
I/O backends. puffs_framebuf is built on the concept of puffs_cc
and uses those to multiplex execution where needed.

File systems are required to implement three methods:
* read frame
* write frame
* compare if frame is a response to the given one

Memory management is provided by puffs_framebuf, but the file
systems must still, of course, interpret the protocol and do e.g.
byte order conversion.

As always, puffs_framebuf is work in progress. Current users are
mount_psshfs and mount_9p.
 1.49  01-May-2007  pooka Introduce PUFFS_FLAG_HASHPATH, which for PUFFS_FLAG_BUILDPATH file servers
creates a hash of the node's path into po->po_hash to avoid a full pathcmp
every time.
 1.48  22-Apr-2007  pooka use more accurate flag masking
 1.47  19-Apr-2007  pooka add puffs_cc_getspecific(pcc), which is shorthand for:
puffs_getspecific(puffs_cc_getusermount(pcc))
 1.46  17-Apr-2007  pooka Introduce PUFFS_STORE_DCOOKIE, which stores a directory cookie for a
given offset iff the directory read in question wants cookies.
 1.45  16-Apr-2007  pooka alloc to set flags when setting fhsize
 1.44  16-Apr-2007  pooka some routines for setting mount parameters
 1.43  13-Apr-2007  pooka Instead of keeping on adding parameters to puffs_mount(), make it
only take the bare essentials, which currently means removing
"maxreqlen" from the argument list (all current callers I'm aware
of set it as 0 anyway). Introduce puffs_init(), which provides a
context for setting various parameters and puffs_domount(), which
can be used to mount the file system. Keep puffs_mount() as a
shortcut for the above two for simple file systems.

Bump development ABI version to 13. After all, it's Friday the 13th.
Watch out! Bad things can happen on Friday the 13th. --No carrier--
 1.42  12-Apr-2007  pooka Hide struct puffs_usermount from the rest of the world and provide
accessors for interesting data in it. Namely, you can now get
pu->pu_privdata with puffs_getspecific(), pu->pu_pn_root with
puffs_set/getroot() and pu->pu_maxreqlen with puffs_getmaxreqlen().
 1.41  11-Apr-2007  pooka * support nodetofh and fhtonode callbacks
* use new signature of node_readdir
 1.40  06-Apr-2007  pooka support page cache flushing
 1.39  06-Apr-2007  pooka implement puffs_inval_pagecache_node_range()
 1.38  22-Mar-2007  pooka unify the location of *pcr in puffs_access_*
 1.37  22-Mar-2007  pooka * return -1 and set errno instead of returning errno directly
* introduce puffs_cred_isregular() which checks if the cred structure
has regular uid/gid (i.e. "person" credentials and not kernel creds)
 1.36  22-Mar-2007  pooka remove variable names from header
 1.35  21-Mar-2007  pooka add routiens which check against ufs semantics for permission to
chown, chmod and change a/mtime
 1.34  20-Mar-2007  pooka implement counterpart of vaccess()
 1.33  20-Mar-2007  pooka initial support for cacheops
 1.32  16-Mar-2007  pooka va->va_mode doesn't contain the full argument to the mknod() system call,
so introduce puffs_addvtype2mode() and use that in null.c to generate the
proper syscall argument
 1.31  15-Feb-2007  pooka Introduce puffs_path_walkcmp(), which is to be called from nodewalk
and compares the path of the node against the given pathobject.
Also make comparison method take a flag to indicate if it should
check if the second path is a true prefix of the first.

plus some namespace cleanup
 1.30  15-Feb-2007  pooka * get rid of the holy kernel-given ISDOTDOT, and instead define a
macro which does strcmp against ".." and (the untranslated)
componentname
* make PUFFS_FLAG_BUILDPATH build paths also if dotdot is the case,
and adapt the regular path objects to this
* make nullfs lookup readable because we can now get rid of dotdot
processing there
 1.29  26-Jan-2007  pooka add interfaces for suspending the file system
 1.28  20-Jan-2007  pooka Be consistent with the rest of the framework about the argument
order of puffs_docc and puffs_dopreq
 1.27  20-Jan-2007  pooka be consistent in naming request handlers
 1.26  16-Jan-2007  pooka revoke revoke
 1.25  15-Jan-2007  pooka Checkpoint some more work in progress: for the benefit those file
system backends which operate purely based on paths, push out more
path management into the library and make path management more
abstract: enable a file system to define a bunch of path management
callbacks, which are used by the framework. Management of normal
/this/is/a/path type paths is provided by the library.
 1.24  11-Jan-2007  pooka implement fsync
 1.23  11-Jan-2007  pooka Add preliminary code for a nullfs layer, which can be used to mount
a directory hierarchy to another point, just like with the kernel
nullfs. This is not really a layering scheme yet, but it should
evolve into one. Currently it can just be used to do 1:1 mapping.
 1.22  10-Jan-2007  pooka Be less shy about the default stack size, it's a only virtual memory space.
TODO: put a guard page at the end
 1.21  09-Jan-2007  pooka rename name cache invalidation interface to be in sync with the
kernel operation names, i.e. contain "namecache" instead of just "name"
 1.20  09-Jan-2007  pooka lib interface for invalidation routines
 1.19  06-Jan-2007  pooka * get rid of the mount callback; it's no great surprise to the
server that it needs to mount the file system backend if it wants
to call mount
* provide some options for getmntopts(), assume that callers will parse
command line (or fstab) args
* reorganize the puffs_cc interface just a bit, preparing for a bigger
revamp later
 1.18  02-Jan-2007  pooka * inform kernel of our version
* homegrown lib ABI check (to avoid bumping major a ludicrillion times
before reaching stability)
 1.17  29-Dec-2006  pooka checkpoint some experimental work-in-progress, namely:

Add support for having multiple outstanding operations. This is done
by exposing enough interfaces so that it is convenient to have the
main event loop in the implementation itself and by providing a
continuation framework for convinient blocking and rescheduling.

works fine, but will undergo further cleanup & development
 1.16  07-Dec-2006  pooka outline & export code for interfacing with kernel GET/PUT requests
 1.15  07-Dec-2006  pooka support mmap
 1.14  07-Dec-2006  pooka rototill the operation callbacks: unify the ops under struct puffs_ops
and namespace them
 1.13  01-Dec-2006  pooka PUFFSFLAG -> PUFFS_FLAG to be consistent with the kernel
 1.12  30-Nov-2006  pooka summon daemon(3) in mainloop unless the nodaemon flag is given
 1.11  23-Nov-2006  pooka const coherence
 1.10  21-Nov-2006  pooka mode_t to enum vtype conversion
 1.9  18-Nov-2006  pooka * call statvfs() at mount() time, pass that info along
* start is gone
 1.8  17-Nov-2006  pooka shuffle flags a bit
 1.7  14-Nov-2006  pooka namespace previous change under puffs_vfsnop to avoid confusions
 1.6  14-Nov-2006  pooka implement unmount, sync and statvfs as dummies for file system which
couldn't care less about implementing them
 1.5  09-Nov-2006  pooka * catch up with kernel changes
* better error handling when mounting
 1.4  07-Nov-2006  pooka * support FAF (Fire-And-Forget) class operations
* adapt to new lookup
 1.3  26-Oct-2006  pooka pass rdev for specfs support
 1.2  25-Oct-2006  pooka handle PUFFS_VN_INACTIVE
 1.1  22-Oct-2006  pooka add libpuffs - the userspace programming interface for puffs

This work was initially started and completed for Google SoC 2005
and tweaked to work a bit better in the past few weeks. While
being far from complete, it is functional enough to be able and
stable to host a fairly general-purpose in-memory file system in
userspace. Even so, puffs should be considered experimental and
no binary compatibility for interfaces or crash-freedom or zero
security implications should be relied upon just yet.

The GSoC project was mentored by William Studenmund and the final
review for the code was done by Christos.
 1.77.4.2  27-Jul-2007  pooka mmap fflags -> prot
 1.77.4.1  27-Jul-2007  pooka file puffs.h was added on branch matt-mips64 on 2007-07-27 08:28:17 +0000
 1.81.2.3  23-Mar-2008  matt sync with HEAD
 1.81.2.2  09-Jan-2008  matt sync with HEAD
 1.81.2.1  06-Nov-2007  matt sync with HEAD
 1.107.6.1  18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.108.8.1  21-Apr-2010  matt sync to netbsd-5
 1.108.4.5  17-Jul-2011  riz Pull up following revision(s) (requested by manu in ticket #1645):
lib/libc/sys/Makefile.inc 1.207 via patch
lib/libc/sys/extattr_get_file.2 patch
lib/libpuffs/dispatcher.c 1.34,1.36 via patch
lib/libpuffs/puffs.c 1.107 via patch
lib/libpuffs/puffs.h 1.115,1.118 via patch
sys/fs/puffs/puffs_msgif.h 1.71,1.76 via patch
sys/fs/puffs/puffs_vfsops.c 1.88 via patch
sys/fs/puffs/puffs_vnops.c 1.145,1.154 via patch
sys/kern/vfs_xattr.c 1.24-1.27 via patch
sys/kern/vnode_if.c 1.87 via patch
sys/sys/Makefile 1.133 via patch
sys/sys/extattr.h 1.6 via patch
sys/sys/vnode_if.h 1.81 via patch
sys/ufs/ffs/ffs_vnops.c patch
sys/ufs/ufs/ufs_extattr.c 1.31,1.34 via patch

* support extended attributes
* bump major due to structure growth
* add some spare space
* remove ABI sillyness
Support extended attributes.
Fix multiple non compliances in our Linux-like extattr API, and make it
public so that it can be used.
Improve a bit listxattr(2). It attemps to list both system and user
extended attributes, and it faled if calling user did not have privilege
for reading system EA. Now we just lise user EA and skip system EA in
reading them is not allowed.
Fix bug introduced in previous commuit: Do not vrele() a vnode we did not
obtained.
Improve UFS1 extended attributes usability
- autocreate attribute backing file for new attributes
- autoload attributes when issuing extattrctl start
- when autoloading attributes, do not display garbage warning when looking
up entries that got ENOENT
Add a flag to VOP_LISTEXTATTR(9) so that the vnode interface can tell the
filesystem in which format extended attribute shall be listed.
There are currently two formats:
- NUL-terminated strings, used for listxattr(2), this is the default.
- one byte length-pprefixed, non NUL-terminated strings, used for
extattr_list_file(2), which is obtanined by setting the
EXTATTR_LIST_PREFIXLEN flag to VOP_LISTEXTATTR(9)
This approach avoid the need for converting the list back and forth, except
in libperfuse, since FUSE uses NUL-terminated strings, and the kernel may
have requested EXTATTR_LIST_PREFIXLEN.
 1.108.4.4  27-Oct-2009  bouyer Pull up following revision(s) (requested by pooka in ticket #1105):
lib/libpuffs/puffs.h: revision 1.112
lib/libpuffs/puffs.c: revision 1.100
distrib/sets/lists/base/md.amd64: patch
distrib/sets/lists/base/md.sparc64: patch
distrib/sets/lists/base/shl.mi: patch
lib/libpuffs/shlib_version: patch

Make it possible to supply parameters to puffs_init() later (but
before puffs_mount()).
 1.108.4.3  18-Oct-2009  sborrill Pull up the following revisions(s) (requested by pooka in ticket #1100):
lib/libpuffs/dispatcher.c: revision 1.33
lib/libpuffs/puffs.c: revision 1.99
lib/libpuffs/puffs.h: revision 1.111
sys/fs/puffs/puffs_msgif.h: revision 1.67 via patch
sys/fs/puffs/puffs_vnops.c: revision 1.136

Support VOP_ABORTOP() in puffs.
 1.108.4.2  16-Oct-2009  sborrill Pull up the following revisions(s) (requested by pooka in ticket #1095):
lib/libpuffs/puffs.c: revision 1.94
lib/libpuffs/puffs.h: revision 1.110

Add puffs_cancel(), which allows to back down after puffs_init()
and before puffs_mount().
 1.108.4.1  16-Oct-2009  sborrill Pull up the following revisions(s) (requested by pooka in ticket #1094):
lib/libpuffs/puffs.3: revision 1.45
lib/libpuffs/puffs.c: revision 1.93
lib/libpuffs/puffs.h: revision 1.109

Add function puffs_setspecific(pu, private) which can be used to set the
specific data after the call to puffs_init().
 1.119.4.5  03-Nov-2014  msaitoh Pull up following revision(s) (requested by manu in ticket #1140):
lib/libperfuse/ops.c 1.63-1.69
lib/libperfuse/perfuse.c 1.32-1.33
lib/libperfuse/perfuse_priv.h 1.32-1.34
lib/libperfuse/subr.c 1.20
lib/libpuffs/creds.c 1.16
lib/libpuffs/dispatcher.c 1.47
lib/libpuffs/puffs.h 1.125
lib/libpuffs/puffs_ops.3 1.37-1.38
lib/libpuffs/requests.c 1.24
sys/fs/puffs/puffs_msgif.h 1.81
sys/fs/puffs/puffs_sys.h 1.85
sys/fs/puffs/puffs_vnops.c 1.183
usr.sbin/perfused/msg.c 1.22
Bring libpuffs, libperfuse and perfused on par with -current:
- implement FUSE direct I/O
- remove useless code and warnings
- fix missing GETATTR bugs
- fix exended attribute get and list operations
 1.119.4.4  13-Sep-2012  riz Pull up following revision(s) (requested by manu in ticket #525):
lib/libpuffs/dispatcher.c: revision 1.44
lib/libp2k/p2k.c: revision 1.57
lib/libpuffs/puffs_ops.3: revision 1.35
lib/libpuffs/pnode.c: revision 1.13
lib/libpuffs/puffs.3: revision 1.56
lib/libpuffs/puffs.h: revision 1.124
Fix regression that has been introduced when the lookup/reclaim race
condition was addressed in libpuffs by counting lookups.
The fix assumes that cookies map to struct puffs_cookie, which has not
been documented as a requirement for filesystems using libpuffs. As an
example, we got burnt by this assumption in libp2k (kern/46734), and
we fixed bit by actually mapping libp2k cookies to struct puffs_node.
It is unlikely, but there may be third party filesystems that use cookies
unmapped to struct puffs_node, and they were left broken for now.
- we introduce a puffs_init() flag PUFFS_FLAG_PNCOOKIE that let filesystems
inform libpuffs that they map cookies to struct puffs_node. Is that flag
is used, the lookup/reclaim race condition fix is enabled. We enable the
flag for libp2k.
- filesystems that use puffs_pn_new() obviouslty use struct puffs_node
and gain PUFFS_FLAG_PNCOOKIE automatically even if they did not specify
it in puffs_init(). This include all our PUFFS filesystem in-tree except
libp2k.
- for filesystems not willing to use struct puffs_node, we introduce a
reclaim2 vnop, which is reclaim with an additionnal lookup count argument.
This vnop let the filesystem implement the lookup/reclaim race fix on
its own.
 1.119.4.3  12-Aug-2012  martin branches: 1.119.4.3.2;
Pull up following revision(s) (requested by manu in ticket #438):
lib/libperfuse/perfuse_priv.h: revision 1.31
sys/fs/puffs/puffs_msgif.h: revision 1.80
sys/fs/puffs/puffs_vnops.c: revision 1.171
lib/libpuffs/puffs_ops.3: revision 1.31
sys/fs/puffs/puffs_vnops.c: revision 1.172
sys/fs/puffs/puffs_vnops.c: revision 1.173
sys/fs/puffs/puffs_vnops.c: revision 1.174
usr.sbin/perfused/perfused.c: revision 1.24
sys/fs/puffs/puffs_sys.h: revision 1.80
sys/fs/puffs/puffs_sys.h: revision 1.81
sys/fs/puffs/puffs_sys.h: revision 1.82
lib/libperfuse/subr.c: revision 1.19
lib/libperfuse/perfuse.c: revision 1.30
sys/fs/puffs/puffs_msgif.c: revision 1.90
sys/fs/puffs/puffs_msgif.c: revision 1.91
sys/fs/puffs/puffs_msgif.c: revision 1.92
lib/libperfuse/ops.c: revision 1.59
lib/libpuffs/puffs.3: revision 1.53
lib/libperfuse/debug.c: revision 1.12
lib/libpuffs/puffs.3: revision 1.54
sys/fs/puffs/puffs_vnops.c: revision 1.167
sys/fs/puffs/puffs_msgif.h: revision 1.79
usr.sbin/perfused/msg.c: revision 1.21
sys/fs/puffs/puffs_vfsops.c: revision 1.102
sys/fs/puffs/puffs_vfsops.c: revision 1.103
sys/fs/puffs/puffs_vfsops.c: revision 1.105
lib/libpuffs/puffs.h: revision 1.123
lib/libperfuse/perfuse_if.h: revision 1.20
lib/libperfuse/perfuse.c: revision 1.29
lib/libpuffs/dispatcher.c: revision 1.42
lib/libpuffs/dispatcher.c: revision 1.43
- Fix same vnodes associated with multiple cookies
The scheme used to retreive known nodes on lookup was flawed, as it only
used parent and name. This produced a different cookie for the same file
if it was renamed, when looking up ../ or when dealing with multiple files
associated with the same name through link(2).
We therefore abandon the use of node name and introduce hashed lists of
inodes. This causes a huge rewrite of reclaim code, which do not attempt
to keep parents allocated until all their children are reclaimed
- Fix race conditions in reclaim
There are a few situations where we issue multiple FUSE operations for
a PUFFS operation. On reclaim, we therefore have to wait for all FUSE
operation to complete, not just the current exchanges. We do this by
introducing node reference count with node_ref() and node_rele().
- Detect data loss caused by FAF
VOP_PUTPAGES causes FAF writes where the kernel does not check the
operation result. At least issue a warning on error.
- Enjoy FAF shortcut on setattr
No need to wait for the result if the kernel does not want it. There is
however an exception for setattr that touch the size, we need to wait
for completion because we have other operations queued for after the
resize.
- Fix fchmod() on write-open file
fchmod() on a node open with write privilege will send setattr with both mode
and size set. This confuses some FUSE filesystem. Therefore we send two FUSE
operations, one for mode, and one for size.
- Remove node TTL handling for netbsd-5 for simplicity sake. The code
still builds on netbsd-5 but does not have the node TTL feature anymore.
It works fine with kernel support on netbsd-6.
- Improve PUFFS_KFLAG_CACHE_FS_TTL by reclaiming older inactive nodes.
The normal kernel behavior is to retain inactive nodes in the freelist
until it runs out of vnodes. This has some merit for local filesystems,
where the cost of an allocation is about the same as the cost of a
lookup. But that situation is not true for distributed filesystems.
On the other hand, keeping inactive nodes for a long time hold memory
in the file server process, and when the kernel runs out of vnodes, it
produce reclaim avalanches that increase lattency for other operations.
We do not reclaim inactive vnodes immediatly either, as they may be
looked up again shortly. Instead we introduce a grace time and we
reclaim nodes that have been inactive beyond the grace time.
- Fix lookup/reclaim race condition.
The above improvement undercovered a race condition between lookup and
reclaim. If we reclaimed a vnode associated with a userland cookie while
a lookup returning that same cookiewas inprogress, then the kernel ends
up with a vnode associated with a cookie that has been reclaimed in
userland. Next operation on the cookie will crash (or at least confuse)
the filesystem.
We fix this by introducing a lookup count in kernel and userland. On
reclaim, the kernel sends the count, which enable userland to detect
situation where it initiated a lookup that is not completed in kernel.
In such a situation, the reclaim must be ignored, as the node is about
to be looked up again.
Fix hang unmount bug introduced by last commit.
We introduced a slow queue for delayed reclaims, while the existing
queue for unmount, flush and exist has been renamed fast queue. Both
queues had timestamp for when an operation should be done, but it was
useless for the fast queue, which is always used to run an operation
ASAP. And the timestamp test had an error that turned ASAP into "at next
tick", but nobody what there to wake the thread at next tick, hence
the hang. The fix is to remove the useless and buggy timestamp test for
fast queue.
Rename slow sopreq queue into node sopreq queue, to refet the fact that
is only intended for postponed node reclaims.
When purging the node sopreq queue, do not call puffs_msg_sendresp(), as
it makes no sense.
Fix race condition between (create|mknod|mkdir|symlino) and reclaim, just
like we did it between lookup and reclaim.
Missing bit in previous commit (prevent race between create|mknod|mkdir|symlink
and reclaim)
Bump date for previous.
New sentence, new line; remove trailing whitespace; fix typos;
punctuation nits.
Add PUFFS_KFLAG_CACHE_DOTDOT so that vnodes hold a reference on their
parent, keeping them active, and allowing to lookup .. without sending
a request to the filesystem.
Enable the featuure for perfused, as this is how FUSE works.
Missing bit in previous commit (PUFFS_KFLAG_CACHE_DOTDOT option to avoid
looking up ..)
 1.119.4.2  05-Jul-2012  riz Pull up following revision(s) (requested by manu in ticket #392):
lib/libpuffs/puffs_ops.3: revision 1.32
lib/libperfuse/ops.c: revision 1.58
lib/libpuffs/puffs.h: revision 1.122
lib/libpuffs/dispatcher.c: revision 1.41
lib/libperfuse/perfuse_priv.h: revision 1.30
Pass the FAF flag to handlers:
- setattr_ttl is updated to add a flag argument. Since it was not
present in
a previous release, we can change its API
- write2 is introduced, this is write with an extra flag for FAF.
- fsync already has the FAF information in a flag and needs no change
- for other operations, FAF is unconditional
Fix the build by adding (unused) flags argument to
perfuse_node_setattr_ttl
 1.119.4.1  23-Apr-2012  riz Pull up following revision(s) (requested by manu in ticket #195):
lib/libskey/skeysubr.c: revision 1.27
lib/libkvm/kvm_getloadavg.c: revision 1.11
lib/libwrap/update.c: revision 1.9
lib/liby/yyerror.c: revision 1.9
lib/libpuffs/puffs_ops.3: revision 1.30
lib/libwrap/misc.c: revision 1.10
lib/libwrap/hosts_access.c: revision 1.20
lib/libpuffs/pnode.c: revision 1.11
lib/libperfuse/subr.c: revision 1.17
lib/libpuffs/pnode.c: revision 1.12
lib/libperfuse/subr.c: revision 1.18
lib/libwrap/options.c: revision 1.15
lib/libwrap/fix_options.c: revision 1.11
lib/libperfuse/ops.c: revision 1.52
lib/libperfuse/ops.c: revision 1.53
lib/libperfuse/ops.c: revision 1.54
lib/libwrap/hosts_ctl.c: revision 1.5
lib/libintl/gettext.c: revision 1.27
lib/libwrap/shell_cmd.c: revision 1.6
lib/libpuffs/dispatcher.c: revision 1.39
lib/libperfuse/perfuse_priv.h: revision 1.27
lib/libwrap/socket.c: revision 1.19
lib/libpuffs/puffs.3: revision 1.50
lib/libperfuse/perfuse_priv.h: revision 1.28
lib/libpuffs/puffs_priv.h: revision 1.45
lib/libpuffs/puffs.3: revision 1.51
lib/libperfuse/perfuse_priv.h: revision 1.29
lib/libwrap/percent_x.c: revision 1.5
lib/libpuffs/puffs.3: revision 1.52
lib/libperfuse/debug.c: revision 1.11
sys/fs/puffs/puffs_vnops.c: revision 1.165
lib/libwrap/tcpd.h: revision 1.13
sys/fs/puffs/puffs_vnops.c: revision 1.166
lib/libwrap/eval.c: revision 1.7
sys/fs/puffs/puffs_msgif.h: revision 1.78
sys/fs/puffs/puffs_vfsops.c: revision 1.101
lib/libwrap/rfc931.c: revision 1.9
lib/libwrap/clean_exit.c: revision 1.5
lib/libpuffs/puffs.h: revision 1.120
lib/libc/stdlib/jemalloc.c: revision 1.27
lib/librmt/rmtlib.c: revision 1.26
lib/libpuffs/puffs.h: revision 1.121
sys/fs/puffs/puffs_sys.h: revision 1.79
lib/librumpclient/rumpclient.c: revision 1.48
lib/libwrap/refuse.c: revision 1.5
lib/libperfuse/perfuse.c: revision 1.26
lib/libperfuse/perfuse.c: revision 1.27
tests/fs/puffs/t_fuzz.c: revision 1.5
lib/libperfuse/perfuse.c: revision 1.28
lib/libpuffs/dispatcher.c: revision 1.40
sys/fs/puffs/puffs_node.c: revision 1.24
lib/libwrap/diag.c: revision 1.9
lib/libintl/textdomain.c: revision 1.13
Use C89 function definition
Add name and atttribute cache with filesytem provided TTL.
lookup, create, mknod, mkdir, symlink, getattr and setattr messages
have been extended so that attributes and their TTL can be provided
by the filesytem. lookup, create, mknod, mkdir, and symlink messages
are also extended so that the filesystem can provide name TTL.
Add PUFFS_KFLAG_CACHE_FS_TTL flag to puffs_init(3) to use name and
attribute cache with filesystem provided TTL.
lookup, create, mknod, mkdir, symlink, getattr and setattr messages
have been extended so that attributes and their TTL can be provided
by the filesytem. lookup, create, mknod, mkdir, and symlink messages
are also extended so that the filesystem can provide name TTL.
The filesystem updates attributes and TTL using
puffs_pn_getvap(3), puffs_pn_getvattl(3), and puffs_pn_getcnttl(3)
Use new PUFFS_KFLAG_CACHE_FS_TTL option to puffs_init(3) so that
FUSE TTL on name and attributes are used. This save many PUFFS
operations and improves performances.
PUFFS_KFLAG_CACHE_FS_TTL is #ifdef'ed in many places for now so that
libperfuse can still be used on netbsd-5.
Split file system.
Comma fixes.
Remove dangling &quot;and&quot;.
Bump date for previous.
- Makesure update_va does not change vnode size when it should not. For
instance when doing a fault-issued VOP_GETPAGES within VOP_WRITE, changing
size leads to panic: genfs_getpages: past eof.
-Handle ticks wrap around for vnode name andattribute timeout
- When using PUFFS_KFLAG_CACHE_FS_TTL, do not use puffs_node to carry
attribute and TTL fora newly created node. Instead extend puffs_newinfo
and add puffs_newinfo_setva() and puffs_newinfo_setttl()
- Remove node_mk_common_final in libperfuse. It used to set uid/gid for
a newly created vnode but has been made redundant along time ago since
uid and gid are properly set in FUSE header.
- In libperfuse, check for corner case where opc = 0 on INACTIVE and RECLAIM
(how is it possible? Check for it to avoid a crash anyway)
- In libperfuse, make sure we unlimit RLIMIT_AS and RLIMIT_DATA so that
we do notrun out of memory because the kernel is lazy at reclaiming vnodes.
- In libperfuse, cleanup style of perfuse_destroy_pn()
Do not set PUFFS_KFLAG_CACHE_FS_TTL for PUFFS tests
 1.119.4.3.2.1  01-Nov-2012  matt sync with netbsd-6-0-RELEASE.
 1.119.2.3  30-Oct-2012  yamt sync with head
 1.119.2.2  23-May-2012  yamt sync with head.
 1.119.2.1  17-Apr-2012  yamt sync with head
 1.124.10.2  05-Nov-2014  snj Pull up following revision(s) (requested by manu in ticket #181):
lib/libperfuse/fuse.h: revision 1.6
lib/libperfuse/ops.c: revision 1.78
lib/libperfuse/perfuse.c: revision 1.35
lib/libperfuse/perfuse_priv.h: revision 1.36
lib/libpuffs/dispatcher.c: revision 1.48
lib/libpuffs/opdump.c: revision 1.37
lib/libpuffs/puffs.c: revision 1.118
lib/libpuffs/puffs.h: revision 1.126
lib/libpuffs/puffs_ops.3: revisions 1.40-1.41
sys/fs/puffs/puffs_msgif.h: revision 1.82-1.83
sys/fs/puffs/puffs_msgif.h: revision 1.82
sys/fs/puffs/puffs_vnops.c: revision 1.196
Add PUFFS support for fallocate and fdiscard operations
--
libpuffs support for fallocate and fdiscard operations
--
Add PUFFS_HAVE_FALLOCATE in puffs_msgif.h so that filesystem can decide
at build time wether fallocate is usable
--
FUSE fallocate support
There seems to be no fdiscard FUSE operation at the moment, hence that
one is left unused.
 1.124.10.1  24-Aug-2014  martin Pull up following revision(s) (requested by manu in ticket #52):
lib/libperfuse/perfuse_priv.h: revision 1.33
lib/libperfuse/ops.c: revision 1.67
lib/libpuffs/dispatcher.c: revision 1.47
lib/libperfuse/perfuse_priv.h: revision 1.34
lib/libperfuse/ops.c: revision 1.68
lib/libperfuse/ops.c: revision 1.69
lib/libpuffs/puffs_ops.3: revision 1.37
lib/libpuffs/puffs_ops.3: revision 1.38
lib/libperfuse/perfuse.c: revision 1.32
lib/libperfuse/perfuse.c: revision 1.33
usr.sbin/perfused/msg.c: revision 1.22
lib/libpuffs/puffs.h: revision 1.125
Implement FUSE direct I/O.
Remove useless code and warnings
 1.127.16.1  13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411

RSS XML Feed