Home | History | Annotate | Download | only in raidframe
History log of /src/sys/dev/raidframe/rf_reconmap.c
RevisionDateAuthorComments
 1.39  08-Apr-2022  andvar s/postion/position/
 1.38  13-Apr-2020  chs slightly change and fix the semantics of pool_set*wat(), pool_sethardlimit()
and pool_prime() (and their pool_cache_* counterparts):

- the pool_set*wat() APIs are supposed to specify thresholds for the count of
free items in the pool before pool pages are automatically allocated or freed
during pool_get() / pool_put(), whereas pool_sethardlimit() and pool_prime()
are supposed to specify minimum and maximum numbers of total items
in the pool (both free and allocated). these were somewhat conflated
in the existing code, so separate them as they were intended.

- change pool_prime() to take an absolute number of items to preallocate
rather than an increment over whatever was done before, and wait for
any memory allocations to succeed. since pool_prime() can no longer fail
after this, change its return value to void and adjust all callers.

- pool_setlowat() is documented as not immediately attempting to allocate
any memory, but it was changed some time ago to immediately try to allocate
up to the lowat level, so just fix the manpage to describe the current
behaviour.

- add a pool_cache_prime() to complete the API set.
 1.37  09-Feb-2019  christos branches: 1.37.10;
- Change the allocation macros to be more like function calls
- Change sizeof(type) -> sizeof(*variable)
- Use macros for the long buffer length allocations
- Remove "bit polishing" memsets() -- do them only once
- Remove unnecessary casts

Thanks to oster@ for finding bugs and testing.
 1.36  14-Nov-2017  christos branches: 1.36.4;
check the result of pool_prime.
 1.35  04-Jan-2017  christos PR/51776: David Binderman: Remove unused variable.
 1.34  20-Feb-2012  oster branches: 1.34.2; 1.34.16; 1.34.20;
comment, and effectively remove, a DIAGNOSTIC check that
is invalid for RAID5_RS.
 1.33  31-Aug-2011  plunky branches: 1.33.2; 1.33.6; 1.33.8;
NULL does not need a cast
 1.32  10-May-2011  mrg convert RF_CommonLogData_s/RF_ReconMap_s mutex to a kmutex/cv.
 1.31  19-May-2008  oster branches: 1.31.8; 1.31.20; 1.31.26;
Re-work some of the guts of the reconstruction code.

Reconmap used to have one pointer for every reconstruction unit. This
does not scale well in the land of 1TB disks, where some 100MB+ of
"status pointers" are required for typical configurations. Convert
the reconstruction code to use a "sliding status window" which will
scale nicely regardless of the number of stripes/reconstruction units
in the RAID set. Convert the main reconstruction loop to rebuild the
array in chunks rather than in one big lump.

As part of these changes, introduce a function to kick any waiters on
the head separation callback list, and use that in the main
reconstruction event queue to wake up the waiters if things have
stalled. (I believe this may fix a race condition that could occur at
at least at the very end of a disk during reconstruction under heavy
IO load.)

Thanks to Brian Buhrow for all his help, support, and patience in
testing these changes.
 1.30  12-Mar-2007  ad branches: 1.30.34; 1.30.36; 1.30.38; 1.30.40;
Pass an ipl argument to pool_init/POOL_INIT to be used when initializing
the pool's lock.
 1.29  16-Nov-2006  christos branches: 1.29.2; 1.29.4; 1.29.8; 1.29.10;
__unused removal on arguments; approved by core.
 1.28  12-Oct-2006  christos - sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386
 1.27  11-Dec-2005  christos branches: 1.27.20; 1.27.22;
merge ktrace-lwp.
 1.26  27-Feb-2005  perry branches: 1.26.2; 1.26.4;
nuke trailing whitespace
 1.25  18-Mar-2004  oster branches: 1.25.8; 1.25.10;
Remove a debugging line that was accidentally left in.
 1.24  18-Mar-2004  oster Re-work the locking mechanisms for reconstruct and PSS structures
such that we don't actually hold a simplelock while we are doing
a pool_get(), but that we still effectively protecting critical code.

This should fix all of the outstanding LOCKDEBUG warnings related to
rebuilding RAID sets.
 1.23  01-Mar-2004  oster We always wait until we get memory, so the NULL check doesn't buy us anything.
 1.22  30-Dec-2003  oster Some days you wonder if some of the function declaration consistency
was just an accident in the first place. Cleanup function decls and
a few comments. [ok.. so I wasn't going to fix this many.. but once
you're on a roll....]
 1.21  29-Dec-2003  oster Since rf_mutex_destroy() is effectively a NOP now, nuke it.
 1.20  29-Dec-2003  oster Simplify defn's of rf_mutex_init() and rf_mutex_destroy(). Since
neither of these ever fail, no need to have a return value. That
makes all the "error detection" on these functions completely
unneeded. But since we're here, if we don't have a return value, then
why not make these macros? My.. look how things keep shrinking, with
no loss in functionality!
 1.19  29-Dec-2003  oster - first kick at a major reworking of RAIDframe's memory allocation code:
- all freelists converted to pools
- initialization of structure members in certain cases where
code was relying on specific allocation and usage properties
to keep structures in a "known state" (that doesn't work with
pools!).
- make most pool_get() be "PR_WAITOK" until they can be analyzed
further, and/or have proper error handling added.
- all RF_Mallocs zero the space returned, so there is no difference
between RF_Calloc and RF_Malloc. In fact, all the RF_Calloc()'s
do is tend to do is get things horribly confused.
Make RF_Malloc() the "general memory allocator", with
RF_MallocAndAdd() the "general memory allocator with
allocation list".
- some of these RF_Malloc's et al. are destined to disappear.
- remove rf_rdp_freelist entirely (it's not used anywhere!)
- remove: #include "rf_freelist.h"
- to the files that were relying on the above, add: #include "rf_general.h"
- add: #include "rf_debugMem.h" to rf_shutdown.h to make it happy
about the loss of: #include "rf_freelist.h".

This shrinks an i386 GENERIC kernel by approx 5K. RAIDframe now
weighs in at about 162K on i386.
 1.18  29-Dec-2003  oster [Having received a definite lack of strenuous objection, a small amount
of strenuous agreement, and some general agreement, this commit is
going ahead because it's now starting to block some other changes I
wish to make.]

Remove most of the support for the concept of "rows" from RAIDframe.
While the "row" interface has been exported to the world, RAIDframe
internals have really only supported a single row, even though they
have feigned support of multiple rows.

Nothing changes in configuration land -- config files still need to
specify a single row, etc. All auto-config structures remain fully
forward/backwards compatible.

The only visible difference to the average user should be a
reduction in the size of a GENERIC kernel (i386) by 4.5K. For those
of us trolling through RAIDframe kernel code, a lot of the driver
configuration code has become a LOT easier to read.
 1.17  21-Dec-2003  oster Since the 5th arg of pool_init() wasn't actually the number
of items with which to prime the pool, we now prime each
pool using pool_prime().
 1.16  21-Dec-2003  simonb Fix usage of fifth argument to pool_init().
 1.15  06-Oct-2002  oster branches: 1.15.6;
poolify the RF_ReconMapListElem_t's used by rf_ReconMapUpdate().
Gives LOCKDEBUG kernels one less thing to complain about.
 1.14  05-Oct-2002  oster update_size() does some bookkeeping, but nothing ever reads the data
it gathers. Nuke it. Without update_size, FreeReconMapListElem() can
do without mapPtr. That, in turn, means crunch_list() doesn't need a
mapPtr either.
 1.13  05-Oct-2002  oster RU_NIL is not used. Remove it.
 1.12  04-Oct-2002  oster Minor cleanup. No functional changes.
 1.11  17-Sep-2002  oster Cast the RF_DEBUG_RECON net a little wider.
 1.10  17-Sep-2002  oster Were these functions actually called by anything, I might have considered
keeping them...
 1.9  14-Sep-2002  oster Everyone and their dog was using RF_ERRORMSG3 to print out the same
sort of error message, over and over again, in different files.
Rather than having the same text repeated in multiple .o files,
create a couple of little functions to do the printing, and save a
bundle of space. Also improves readability of code.
 1.8  13-Nov-2001  lukem add RCSIDs
 1.7  18-Jul-2001  thorpej bzero -> memset
 1.6  14-Aug-1999  oster branches: 1.6.14; 1.6.16;
Fix an 'off-by-one' error in a RF_ASSERT which meant that a reconstruct
would panic in the case where the stripe width was 1 block.
 1.5  13-Aug-1999  oster rf_sys.h does not need to be #included in any of these files, and, actually,
is no longer needed at all.
 1.4  05-Feb-1999  oster branches: 1.4.2;
Phase 2 of the RAIDframe cleanup. The source is now closer to KNF
and is much easier to read. No functionality changes.
 1.3  26-Jan-1999  oster Nuke more bits of RAIDframe "demo" code. We're not "demoing" here,
we're doing the Real Thing!
 1.2  26-Jan-1999  oster RAIDframe cleanup, phase 1. Nuke simulator support, user-land driver,
out-dated comments, and other unneeded stuff. This helps prepare
for cleaning up the rest of the code, and adding new functionality.

No functional changes to the kernel code in this commit.
 1.1  13-Nov-1998  oster RAIDframe, version 1.1, from the Parallel Data Laboratory at
Carnegie Mellon University. Full RAID implementation, including
levels 0, 1, 4, 5, 6, parity logging, and a few other goodies.
Ported to NetBSD by Greg Oster.
 1.4.2.1  28-Sep-1999  cgd pull up rev 1.6 from trunk (requested by oster):
Fix an 'off-by-one' error in a RF_ASSERT which meant that a reconstruct
would panic in the case where the stripe width was 1 block.
 1.6.16.3  10-Oct-2002  jdolecek sync kqueue with -current; this includes merge of gehenna-devsw branch,
merge of i386 MP branch, and part of autoconf rototil work
 1.6.16.2  10-Jan-2002  thorpej Sync kqueue branch with -current.
 1.6.16.1  03-Aug-2001  lukem update to -current
 1.6.14.4  18-Oct-2002  nathanw Catch up to -current.
 1.6.14.3  17-Sep-2002  nathanw Catch up to -current.
 1.6.14.2  14-Nov-2001  nathanw Catch up to -current.
 1.6.14.1  24-Aug-2001  nathanw Catch up with -current.
 1.15.6.4  04-Mar-2005  skrll Sync with HEAD.

Hi Perry!
 1.15.6.3  21-Sep-2004  skrll Fix the sync with head I botched.
 1.15.6.2  18-Sep-2004  skrll Sync with HEAD.
 1.15.6.1  03-Aug-2004  skrll Sync with HEAD
 1.25.10.1  19-Mar-2005  yamt sync with head. xen and whitespace. xen part is not finished.
 1.25.8.1  29-Apr-2005  kent sync with -current
 1.26.4.1  03-Sep-2007  yamt sync with head.
 1.26.2.1  25-May-2008  bouyer Pull up following revision(s) (requested by oster in ticket #1933):
sys/dev/raidframe/rf_reconmap.h: revision 1.11
sys/dev/raidframe/rf_reconmap.c: revision 1.31
sys/dev/raidframe/rf_reconstruct.h: revision 1.24
sys/dev/raidframe/rf_reconstruct.c: revision 1.104
sys/dev/raidframe/rf_revent.c: revision 1.25
Convert the reconstruction code to use a "sliding status window" which will
scale nicely regardless of the number of stripes/reconstruction units
in the RAID set. Convert the main reconstruction loop to rebuild the
array in chunks rather than in one big lump. May fix bin/38471.
 1.27.22.2  10-Dec-2006  yamt sync with head.
 1.27.22.1  22-Oct-2006  yamt sync with head
 1.27.20.1  18-Nov-2006  ad Sync with head.
 1.29.10.1  03-Jun-2008  skrll Sync with netbsd-4.
 1.29.8.1  13-Mar-2007  ad Sync with head.
 1.29.4.1  24-Mar-2007  yamt sync with head.
 1.29.2.1  25-May-2008  bouyer Pull up following revision(s) (requested by oster in ticket #1153):
sys/dev/raidframe/rf_reconmap.h: revision 1.11
sys/dev/raidframe/rf_reconmap.c: revision 1.31
sys/dev/raidframe/rf_reconstruct.h: revision 1.24
sys/dev/raidframe/rf_reconstruct.c: revision 1.104
sys/dev/raidframe/rf_revent.c: revision 1.25
Convert the reconstruction code to use a "sliding status window" which will
scale nicely regardless of the number of stripes/reconstruction units
in the RAID set. Convert the main reconstruction loop to rebuild the
array in chunks rather than in one big lump. May fix bin/38471.
 1.30.40.1  23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.30.38.1  04-May-2009  yamt sync with head.
 1.30.36.1  04-Jun-2008  yamt sync with head
 1.30.34.1  02-Jun-2008  mjf Sync with HEAD.
 1.31.26.1  06-Jun-2011  jruoho Sync with HEAD.
 1.31.20.1  31-May-2011  rmind sync with head
 1.31.8.1  24-Feb-2012  sborrill Pull up the following revisions(s) (requested by oster in ticket #1728):
sys/dev/raidframe/rf_reconmap.c: revision 1.34
sys/dev/raidframe/rf_reconstruct.c: revision 1.118

Remove a DIAGNOSTIC check that is invalid for RAID5_RS.
Add logic to the main reconstruction loop to handle RAID5 with rotated
spares. Correct issue where we were doing one more stripe than necessary.
 1.33.8.1  23-Feb-2012  riz Pull up following revision(s) (requested by oster in ticket #23):
sys/dev/raidframe/rf_reconstruct.c: revision 1.118
sys/dev/raidframe/rf_reconmap.c: revision 1.34
comment, and effectively remove, a DIAGNOSTIC check that
is invalid for RAID5_RS.
Add logic to the main reconstruction loop to handle RAID5 with rotated
spares. While here, observe that we were actually doing one more
stripe than we thought we were, and correct that too (it didn't matter
for non-RAID5_RS, but it definitely does for RAID5_RS). Add some
bounds-checking at the beginning to handle the case where the number
of stripes in the set is smaller than the sliding reconstruction window.
XXX: this problem likely needs to be fixed for PARITY_DECLUSTERING too.
 1.33.6.1  24-Feb-2012  mrg sync to -current.
 1.33.2.1  17-Apr-2012  yamt sync with head
 1.34.20.1  07-Jan-2017  pgoyette Sync with HEAD. (Note that most of these changes are simply $NetBSD$
tag issues.)
 1.34.16.1  05-Feb-2017  skrll Sync with HEAD
 1.34.2.1  03-Dec-2017  jdolecek update from HEAD
 1.36.4.2  21-Apr-2020  martin Sync with HEAD
 1.36.4.1  10-Jun-2019  christos Sync with HEAD
 1.37.10.1  20-Apr-2020  bouyer Sync with HEAD

RSS XML Feed