Home | History | Annotate | Download | only in raidframe
History log of /src/sys/dev/raidframe/rf_engine.c
RevisionDateAuthorComments
 1.53  10-Oct-2019  christos fix the function pointer and callback mess:
- callback functions return 0 and their result is not checked; make them void.
- there are two types of callbacks and they used to overload their parameters
and the callback structure; separate them into "function" and "value"
callbacks.
- make the wait function signature consistent.
 1.52  11-Dec-2016  nat branches: 1.52.16;
Ensure variables are initialized. Fixes error due to -Wmaybe-unitialized.
 1.51  03-Jan-2016  mlelstv branches: 1.51.2;
refactor driver to use common code in dksubr.
 1.50  18-Oct-2014  snj branches: 1.50.2;
src is too big these days to tolerate superfluous apostrophes. It's
"its", people!
 1.49  15-Sep-2013  martin ifdef variables like their use
 1.48  27-Apr-2013  christos branches: 1.48.4;
allocate devices dynamically.
 1.47  07-Sep-2011  mbalmer branches: 1.47.2; 1.47.12;
Newline is \n, not /n.
 1.46  11-May-2011  mrg convert the main raidPtr mutex to a kmutex, and add a couple of cv's to
cover the old sleep/wakeup points for adding_hot_spare and waitForReconCond.
convert all remaining simple_lock's to kmutexes (they're not used or compiled
right now... even with all options enabled) and remove the support for them.

this leaves just a pair of tsleep()/wakeup() calls using old scheduling APIs.
 1.45  02-May-2011  mrg use a unique wchan identifier for node_queue_cv than the old default.
 1.44  27-Apr-2011  mrg prepare to convert more raidframe old lock/sleep APIs to mutex/condvar:

- remove RF_DECLARE_EXTERN_MUTEX and RF_DECLARE_STATIC_MUTEX, the qualifier
can be provided at the use point with the normal define
- rename the *LGMGR_MUTEX() macros to *mutex2() names, and add some more
defines for use:
rf_declare_mutex2()
rf_declare_cond2()
rf_lock_mutex2()
rf_unlock_mutex2()
rf_init_mutex2()
rf_destroy_mutex2()
rf_init_cond2()
rf_destroy_cond2()
rf_wait_cond2()
rf_signal_cond2()
rf_broadcast_cond2()
- use the new names for the configureMutex(), which previous used some combo
of direct mutex* calls and macros
- convert the node_queue to use a mutex/cv combo
- in rf_ShutdownEngine() and DAGExecutionThread(), also signal the former from
the latter when it is done and about to exit
- convert iodone_lock to use the new macros
 1.43  23-Apr-2011  mrg move the iodone setup and tear down into rf_engine.c. this fixes
"raidctl -u" and should also help the case where we fail to setup
a device part of the way through.
 1.42  23-Apr-2011  mrg convert the iodone_lock to a mutex, and use a condvar for signalling.

this only handles the smallest use of old simple_lock/tsleep/wakeup
APIs inside raidframe, and it points out that cv(9)'s have only one
wait channel per cv, whereas each tsleep() caller can specify a
different wait channel. this change removes the difference between
normal raidio and waiting for IO during shutdown.

i've tested this one 3 systems, ran atf, and had mlelstv and rmind
review the change.
 1.41  13-Sep-2010  drochner branches: 1.41.2;
comment out an unused static function
 1.40  17-Nov-2009  jld branches: 1.40.2; 1.40.4;
Finally commit the RAIDframe parity map Summer Of Code project.

Drastically reduces the amount of time spent rewriting parity after an
unclean shutdown by keeping better track of which regions might have had
outstanding writes. Enabled by default; can be disabled on a per-set
basis, or tuned, with the new raidctl(8) commands.

Discussed on tech-kern@ to a general air of approval; exhortations to
commit from mrg@, christos@, and others.

Thanks to Google for their sponsorship, oster@ for mentoring the
project, assorted developers for trying very hard to break it, and
probably more I'm forgetting.
 1.39  16-Nov-2006  christos branches: 1.39.52; 1.39.64; 1.39.72;
__unused removal on arguments; approved by core.
 1.38  12-Oct-2006  christos - sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386
 1.37  11-Dec-2005  christos branches: 1.37.20; 1.37.22;
merge ktrace-lwp.
 1.36  25-Sep-2005  oster Re-work the handling of incoming I/O in RAIDframe:
- introduce rf_buf_queue_check() which checks to see if there
is work to do in the incoming buffer queue
- rf_RaidIOThread() is now responsible for calling raidstart(), and is
also now the only place that calls raidstart()
- raidstrategy() now just queues requests in buf_queue
and signals rf_RaidIOThread() that work has arrived

Hopefully addresses PR#30233
 1.35  27-Feb-2005  perry branches: 1.35.2; 1.35.4;
nuke trailing whitespace
 1.34  09-Mar-2004  oster branches: 1.34.8; 1.34.10;
Introduce RF_DEBUG_ENGINE and use it to disable a number of debugging
bits in rf_engine.c
 1.33  01-Mar-2004  oster Use RF_ACC_TRACE to #if out more chunks of code related only
to access tracing. (not turned on yet)
 1.32  29-Feb-2004  oster Adjust _rf_ShutdownCreate() so that it is willing to wait for more
memory. Since we only now ever "return(0)", just return (void)
instead.

Cleanup all uses of rf_ShutdownCreate() to not worry about
it ever failing. Shaves another 600 bytes off of an i386 GENERIC kernel.
 1.31  02-Jan-2004  oster Fix the "We panic if we can't create a DAG" problem that's existed
~forever. This requires a number of things:

1) If we can't create a DAG, set desc->numStripes to 0 in
rf_SelectAlgorithm. This will ensure that we don't attempt to free
any dagArray[] elements in rf_StateCleanup.

2) Modify rf_State_CreateDAG() to not panic in the event of a DAG
failure. Instead, set the bp->b_flags and bp->b_error, and set things
up to skip to rf_State_Cleanup().

3) Need to mark desc->status as "bad" so that we actually stop looking
for a different DAG. (which we won't find... no matter how many times
we try).

4) rf_State_LastState() will then do the biodone(), and return EIO for
the IO in question.

5) Remove some " || 1 "'s from ProcessNode(). These were for
debugging, and we don't need the failure notices spewing
over and over again as the failing DAGs are processed.

6) Needed to change

if (asmap->numDataFailed + asmap->numParityFailed > 1)

to

if ((asmap->numDataFailed + asmap->numParityFailed > 1) ||
(raidPtr->numFailures > 1)){

in rf_raid5.c so that it doesn't try to return
rf_CreateNonRedundantWriteDAG as the creation function.

7) Note that we can't apply the above change to the RAID 1 code as
with the silly "fake 2-D" RAID 1 sets, it is possible to have 2 failed
components in the RAID 1 set, and that would stop them from working.
(I really don't know why/how those "fake 2-D" RAID 1 sets even work
with all the "single-fault" assumptions present in the rest of the
code.)

8) Needed to protect rf_RAID0DagSelect() in a similar way -- it should
return NULL as the createFunc.

9) No point printing out "Multiple disks failed..." a zillion times.
 1.30  01-Jan-2004  oster Nuke a bunch of unused variables:
- node_queue_cond
- quiescent_cond
- eq_cond
- desc->cond
- desc->head
- diskqueue->numWaiting

Nuke rf_print_unable_to_init_cond().
Nuke rf_TerminateDiskQueues prototype from rf_diskqueue.h.
 1.29  30-Dec-2003  oster Some days you wonder if some of the function declaration consistency
was just an accident in the first place. Cleanup function decls and
a few comments. [ok.. so I wasn't going to fix this many.. but once
you're on a roll....]
 1.28  29-Dec-2003  oster Garbage-collect a whole mess of this RF_THREADGROUP_* stuff that isn't
being used. Then, nuke rf_init_managed_threadgroup() and all
descendants and relations.
 1.27  29-Dec-2003  oster _rf_create_managed_cond() is now left doing nothing. Convert callers.
Mash DO_RAID_COND in rf_driver.c out of existance.

- Nuke (already #if 0'ed) _rf_create_managed_lkmgr_mutex() while we're
busy here.

simplify DO_INIT in rf_engine.c
 1.26  29-Dec-2003  oster _rf_create_managed_mutex() is doing just a simple:

rf_mutex_init(m)

now. The rest of the fluff is no longer needed.
It also cannot fail, so error checking on rf_create_managed_mutex()
is just wasting space.

Nuke the #define's associated with rf_create_managed_mutex().
Convert rf_create_managed_mutex(listp,m) to just rf_mutex_init(m).
Remove wasteful "error checking" and simplify all instances where this
is called. (another 0.3K saved in the binary, but the real savings
is in code readability!)
 1.25  04-Oct-2002  oster branches: 1.25.6;
Clean up #includes. Group function prototypes. Fix more 80-column lossage
that was missed before. Cleanup a printf.
 1.24  04-Oct-2002  oster Undo the change from 1.16->1.17, and 1.19->1.20. The ltsleep and locking
that was there before before is just fine, and plays nicely with LOCKDEBUG
now that we've added the raidio thread.
 1.23  04-Oct-2002  oster Tidy up comments and do some 80-column fixing. No functional changes.
 1.22  04-Oct-2002  oster As part of the effort to get RAIDframe playing nicely with LOCKDEBUG,
rework how completed requests are handled. In particular, instead of
doing all sorts of work and locking in interrupt context, completed
requests are now queued. A new kernel thread (rf_RaidIOThread) now
handles calling rf_DiskIOComplete() and (req->CompleteFunc)() for each
completed request. There is still work to be done to make RAIDframe
LOCKDEBUG friendly, but this change is a huge step forward.

Reviewed by (and many thanks to): thorpej
 1.21  02-Oct-2002  oster Create a thread creation macro that make it easier to name the DAG process.
Use it to have the DAGExecutionThread have a better name for each RAID
process ('raid0', 'raid1', etc., vs. just 'raid', 'raid', etc.)
 1.20  02-Oct-2002  oster Since we are unlocking the lock before sleeping, there is no need
to use ltsleep() when a tsleep() will suffice. Makes RAIDframe+LOCKDEBUG
play together much better.
 1.19  23-Sep-2002  oster Nuke a couple of unreached 'break's. Thanks (again) to Simon B.
 1.18  19-Sep-2002  oster Nuke an obsolete comment.
 1.17  15-Sep-2002  oster Release our hold on raidPtr if we're going to sleep.
 1.16  14-Sep-2002  oster Everyone and their dog was using RF_ERRORMSG3 to print out the same
sort of error message, over and over again, in different files.
Rather than having the same text repeated in multiple .o files,
create a couple of little functions to do the printing, and save a
bundle of space. Also improves readability of code.
 1.15  14-Jul-2002  oster Remove incorrect comment.
 1.14  13-Jul-2002  oster Most folks won't need the DAG printing and verification routines.
Introduce a #define to toggle them on/off. Disable calls to
rf_PrintDAGList(). Saves ~6K on GENERIC+DEBUG kernel on i386.
 1.13  13-Jul-2002  oster rf_ValidateDAG isn't something that we want to even consider
calling unless DEBUG is turned on.
 1.12  13-Nov-2001  lukem branches: 1.12.8;
add RCSIDs
 1.11  16-Sep-2001  wiz Spell 'occurred' with two 'r's.
 1.10  20-Aug-2000  thorpej branches: 1.10.2; 1.10.4; 1.10.6;
Use RF_WAIT_COND() to avoid locking errors.
 1.9  08-Jan-2000  oster - nuke calls to rf_get_threadid() and associated #include
- change a bunch of debugging printfs from
"[%d] ...", tid (where tid is the "thread id")
to
"raid%d: ...", raidPtr->raidid
- other minor rototillage
 1.8  08-Jan-2000  oster Minor cleanup.
 1.7  07-Jan-2000  oster Nuke RF_DECLARE_GLOBAL_THREADID, rf_setup_threadid(), rf_shutdown_threadid(),
and rf_assign_threadid(). rf_threadid.h, your days are numbered.
 1.6  05-Jan-2000  oster - update RF_CREATE_THREAD to handle a 'process name' argument.
- fire up a new thread for parity re-writes, copybacks, and reconstructs.
The ioctl's which trigger these actions now return immediately.
- add progress accounting for the above actions.
- minor rototillage of rf_netbsdkintf.c to deal with all of the above.
 1.5  14-Mar-1999  oster branches: 1.5.8;
Disallow interrupting parity re-writes or copybacks by removing PCATCH
from the tsleep()'s (they probably shouldn't have been there in the
first place!). Making parity re-writing and copybacks interruptable
will require re-designing how a few things are done (e.g. how memory
is freed for structures shipped off to routines that run asynchronously
relative to the calling routine). Fix a few other tsleep's while we're at it.
 1.4  05-Feb-1999  oster Phase 2 of the RAIDframe cleanup. The source is now closer to KNF
and is much easier to read. No functionality changes.
 1.3  26-Jan-1999  oster RAIDframe cleanup, phase 1. Nuke simulator support, user-land driver,
out-dated comments, and other unneeded stuff. This helps prepare
for cleaning up the rest of the code, and adding new functionality.

No functional changes to the kernel code in this commit.
 1.2  13-Nov-1998  simonb Add braces around 'if' to pacify egcs
 1.1  13-Nov-1998  oster RAIDframe, version 1.1, from the Parallel Data Laboratory at
Carnegie Mellon University. Full RAID implementation, including
levels 0, 1, 4, 5, 6, parity logging, and a few other goodies.
Ported to NetBSD by Greg Oster.
 1.5.8.1  20-Nov-2000  bouyer Update thorpej_scsipi to -current as of a month ago
A i386 GENERIC kernel compiles without the siop, ahc and bha drivers
(will be updated later). i386 IDE/ATAPI and ncr work, as well as
sparc/esp_sbus. alpha should work as well (untested yet).
siop, ahc and bha will be updated once I've updated the branch to current
-current, as well as machine-dependant code.
 1.10.6.1  01-Oct-2001  fvdl Catch up with -current.
 1.10.4.3  10-Oct-2002  jdolecek sync kqueue with -current; this includes merge of gehenna-devsw branch,
merge of i386 MP branch, and part of autoconf rototil work
 1.10.4.2  06-Sep-2002  jdolecek sync kqueue branch with HEAD
 1.10.4.1  10-Jan-2002  thorpej Sync kqueue branch with -current.
 1.10.2.5  18-Oct-2002  nathanw Catch up to -current.
 1.10.2.4  17-Sep-2002  nathanw Catch up to -current.
 1.10.2.3  01-Aug-2002  nathanw Catch up to -current.
 1.10.2.2  14-Nov-2001  nathanw Catch up to -current.
 1.10.2.1  21-Sep-2001  nathanw Catch up to -current.
 1.12.8.1  15-Jul-2002  gehenna catch up with -current.
 1.25.6.5  10-Nov-2005  skrll Sync with HEAD. Here we go again...
 1.25.6.4  04-Mar-2005  skrll Sync with HEAD.

Hi Perry!
 1.25.6.3  21-Sep-2004  skrll Fix the sync with head I botched.
 1.25.6.2  18-Sep-2004  skrll Sync with HEAD.
 1.25.6.1  03-Aug-2004  skrll Sync with HEAD
 1.34.10.1  19-Mar-2005  yamt sync with head. xen and whitespace. xen part is not finished.
 1.34.8.1  29-Apr-2005  kent sync with -current
 1.35.4.1  21-Jun-2006  yamt sync with head.
 1.35.2.1  04-Oct-2005  tron Pull up following revision(s) (requested by oster in ticket #853):
sys/dev/raidframe/rf_netbsdkintf.c: revision 1.190
sys/dev/raidframe/rf_netbsd.h: revision 1.24
sys/dev/raidframe/rf_states.c: revision 1.39
sys/dev/raidframe/rf_engine.c: revision 1.36
Re-work the handling of incoming I/O in RAIDframe:
- introduce rf_buf_queue_check() which checks to see if there
is work to do in the incoming buffer queue
- rf_RaidIOThread() is now responsible for calling raidstart(), and is
also now the only place that calls raidstart()
- raidstrategy() now just queues requests in buf_queue
and signals rf_RaidIOThread() that work has arrived
Hopefully addresses PR#30233
 1.37.22.2  10-Dec-2006  yamt sync with head.
 1.37.22.1  22-Oct-2006  yamt sync with head
 1.37.20.1  18-Nov-2006  ad Sync with head.
 1.39.72.1  21-Apr-2010  matt sync to netbsd-5
 1.39.64.1  10-Dec-2009  snj Pull up following revision(s) (requested by tron in ticket #1187):
sbin/raidctl/raidctl.8: revisions 1.57-1.59 via patch
sbin/raidctl/raidctl.c: revision 1.42 via patch
sys/dev/raidframe/files.raidframe: revision 1.8 via patch
sys/dev/raidframe/rf_copyback.c: revision 1.42 via patch
sys/dev/raidframe/rf_disks.c: revision 1.72 via patch
sys/dev/raidframe/rf_driver.c: revision 1.122 via patch
sys/dev/raidframe/rf_engine.c: revision 1.40 via patch
sys/dev/raidframe/rf_kintf.h: revision 1.21 via patch
sys/dev/raidframe/rf_netbsdkintf.c: revision 1.269 via patch
sys/dev/raidframe/rf_paritymap.c: revisions 1.1-1.3 via patch
sys/dev/raidframe/rf_paritymap.h: revision 1.1 via patch
sys/dev/raidframe/rf_parityscan.c: revision 1.33 via patch
sys/dev/raidframe/rf_parityscan.h: revision 1.8 via patch
sys/dev/raidframe/rf_raid.h: revision 1.38 via patch
sys/dev/raidframe/rf_reconstruct.c: revision 1.108 via patch
sys/dev/raidframe/rf_states.c: revision 1.44 via patch
sys/dev/raidframe/raidframeio.h: revision 1.6 via patch
sys/dev/raidframe/raidframevar.h: revision 1.13 via patch
Pull up the RAIDframe parity map Summer Of Code project.
Drastically reduces the amount of time spent rewriting parity after an
unclean shutdown by keeping better track of which regions might have had
outstanding writes. Enabled by default; can be disabled on a per-set
basis, or tuned, with the new raidctl(8) commands.
 1.39.52.2  09-Oct-2010  yamt sync with head
 1.39.52.1  11-Mar-2010  yamt sync with head
 1.40.4.2  31-May-2011  rmind sync with head
 1.40.4.1  05-Mar-2011  rmind sync with head
 1.40.2.1  22-Oct-2010  uebayasi Sync with HEAD (-D20101022).
 1.41.2.1  06-Jun-2011  jruoho Sync with HEAD.
 1.47.12.3  03-Dec-2017  jdolecek update from HEAD
 1.47.12.2  20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.47.12.1  23-Jun-2013  tls resync from head
 1.47.2.1  22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.48.4.1  18-May-2014  rmind sync with head
 1.50.2.2  05-Feb-2017  skrll Sync with HEAD
 1.50.2.1  19-Mar-2016  skrll Sync with HEAD
 1.51.2.1  07-Jan-2017  pgoyette Sync with HEAD. (Note that most of these changes are simply $NetBSD$
tag issues.)
 1.52.16.1  13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411

RSS XML Feed