Home | History | Annotate | Download | only in raidframe
History log of /src/sys/dev/raidframe/rf_map.c
RevisionDateAuthorComments
 1.51  23-Jul-2021  oster Extensive mechanical changes to the pools used in RAIDframe.

Alloclist remains not per-RAID, so initialize that pool
separately/differently than the rest.

The remainder of pools in RF_Pools_s are now per-RAID pools. Mostly
mechanical changes to functions to allocate/destroy per-RAID pools.
Needed to make raidPtr available in certain cases to be able to find
the per-RAID pools.

Extend rf_pool_init() to now populate a per-RAID wchan value that is
unique to each pool for a given RAID device.

TODO: Complete the analysis of the minimum number of items that are
required for each pool to allow IO to progress (i.e. so that a request
for pool resources can always be satisfied), and dynamically scale
minimum pool sizes based on RAID configuration.
 1.50  28-May-2019  msaitoh branches: 1.50.14;
s/recieve/receive/
 1.49  10-Feb-2019  christos Introduce PR_ZERO to avoid open-coding memset()s everywhere. OK riastradh@.
 1.48  09-Feb-2019  christos - Change the allocation macros to be more like function calls
- Change sizeof(type) -> sizeof(*variable)
- Use macros for the long buffer length allocations
- Remove "bit polishing" memsets() -- do them only once
- Remove unnecessary casts

Thanks to oster@ for finding bugs and testing.
 1.47  15-Oct-2016  oster branches: 1.47.16;

Cleanup/remove unused asm_count. Reported by David Binderman in PR#51552.
Thanks!
 1.46  14-Nov-2014  oster branches: 1.46.2; 1.46.4;


Adjust comment to reflect reality. (5th arg, not 4th)
 1.45  11-May-2011  mrg branches: 1.45.14;
convert the main raidPtr mutex to a kmutex, and add a couple of cv's to
cover the old sleep/wakeup points for adding_hot_spare and waitForReconCond.
convert all remaining simple_lock's to kmutexes (they're not used or compiled
right now... even with all options enabled) and remove the support for them.

this leaves just a pair of tsleep()/wakeup() calls using old scheduling APIs.
 1.44  15-Mar-2009  cegger branches: 1.44.4; 1.44.6;
ansify function definitions
 1.43  04-Mar-2007  christos branches: 1.43.40; 1.43.50; 1.43.56;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.
 1.42  16-Nov-2006  christos branches: 1.42.4;
__unused removal on arguments; approved by core.
 1.41  12-Oct-2006  christos - sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386
 1.40  26-Apr-2006  oster branches: 1.40.8; 1.40.10;
un-bitrot some of this code (which is not included by default in kernels),
and make it build again. XXX: Paritylogging bits still won't build.
 1.39  11-Dec-2005  christos branches: 1.39.4; 1.39.6; 1.39.8; 1.39.10; 1.39.12;
merge ktrace-lwp.
 1.38  27-Feb-2005  perry branches: 1.38.4;
nuke trailing whitespace
 1.37  20-Mar-2004  oster branches: 1.37.8; 1.37.10;
For each RAID set, pre-allocate a number of "emergency buffers" to be
used in the event that we can't malloc a buffer of the appropriate
size in the traditional way. rf_AllocIOBuffer() and rf_FreeIOBuffer()
deal with allocating/freeing these structures. These buffers are
stored in a list on the 'iobuf' list. iobuf_count keeps track of how
many buffers are available, and numEmergencyBuffers is the effective
"high-water" mark for the freelist. The buffers allocated by
rf_AllocIOBuffer() are stripe-unit sized, which is the maximum
size requested by any of the callers.

Add an iobufs entry to RF_DagHeader_s. Use it for keeping track of
buffers that get allocated from the free-list.

Add a "generic list" pool (VoidPointerListElement Pool) for elements
used to maintain a list of allocated memory. [It is somewhat less
than ideal to add another little pool to handle this...]

Teach rf_AllocBuffer() to use the new rf_AllocIOBuffer(). Modify
other Mallocs to use rf_AllocIOBuffer(), and to update dag_h->iobufs as
appropriate.

Update rf_FreeDAG() to handle cleanup of dag_h->iobufs.

While here, add some missing pool_destroy() calls for a number of pools.

With these changes, it should (in theory) be possible to swap on
RAID 5 sets again. That said, I've not had any success there yet --
but the last issue I saw at least wasn't in RAIDframe. :-}

[There is room for this code to become a bit more consise, but I
wanted to do a checkpoint here with something known to work :) ]
 1.36  19-Mar-2004  oster Take care of six more mallocs:

- Pull rf_FreePhysDiskAddr() out from under a #ifdef, since we're now
going to use it.

- Add a pda_cleanup_list into the DAG header. Use it in rf_FreeDAG() to
cleanup any PDA's that get allocated but have no "easy" way of being
located and freed when the DAG completes.

- numStripeUnitsAccessed is a per-stripe value, and has a maximum
value equal to the number of colums (thus limited by RF_MAXCOL).
Use this knowledge to set a high-bound on overlappingPDAs, and stuff
it on the stack instead of malloc'ing it all the time! This costs us
a whopping 40 bytes on the stack, but saves a malloc() and a free().
 1.35  19-Mar-2004  oster Add a comment. Will hopefully save time next time someone tries
to figure out where the allocated memory is freed.
 1.34  19-Mar-2004  oster Introduce 3 more pools and 6 functions to handle allocating/freeing
elements from the pools.

Re-work rf_SelectAlgorithm() to get rid of all the 8 malloc's, and to
use the new functions to get/put these 'support structures'. I'm not
overly happy with some of the variable names, but them's the breaks.

In the process of changing things, fix a bug:
- in the case where we can't create a dag, free asmh_b and blockFuncs
too!!

[if you were able to look at the source code related to these changes,
and comprehend what was going on without having your eyes bleed or
getting dizzy, please contact me... I'm sure I'll have more code
which would benefit by you having a look at it before I commit it :) ]
 1.33  07-Mar-2004  oster - Introduce rf_pools which contains all of the various global pools used
by RAIDframe. Convert all other RAIDframe global pools to use pools
defined within this new structure.
- Introduce rf_pool_init(), used for initializing a single pool in
RAIDframe. Teach each of the configuration routines to use
rf_pool_init().
- Cleanup a few pool-related comments.
- Cleanup revent initialization and #defines.
- Add a missing pool_destroy() for the reconbuffer pool.

(Saves another 1K off of an i386 GENERIC kernel, and makes
stuff a lot more readable)
 1.32  07-Mar-2004  oster - Introduce rf_pools which contains all of the various global pools used
by RAIDframe. Convert all other RAIDframe global pools to use pools
defined within this new structure.
- Introduce rf_pool_init(), used for initializing a single pool in
RAIDframe. Teach each of the configuration routines to use
rf_pool_init().
- Cleanup a few pool-related comments.
- Cleanup revent initialization and #defines.
- Add a missing pool_destroy() for the reconbuffer pool.

(Saves another 1K off of an i386 GENERIC kernel, and makes
stuff a lot more readable)
 1.31  07-Mar-2004  oster We don't need 2-fault tollerant stuff here in the default case.
Use #if's to conditionalize the code compilation.
(Saves another 1K on i386 GENERIC kernels)
 1.30  05-Mar-2004  oster - remove the RF_*_INC's, as necessary. They are not needed any more.
- introduce RF_MIN_*'s, as necessary. These will indicate the
low-water mark for pools as well as the pool_prime() value.
- add pool_setlowat() for the critical pools.
- pool_prime() and pool_setlowat() the raidframe_cbufpool.
- re-order some pool_prime()'s and pool_sethiwat()'s for clarity.
 1.29  29-Feb-2004  oster Adjust _rf_ShutdownCreate() so that it is willing to wait for more
memory. Since we only now ever "return(0)", just return (void)
instead.

Cleanup all uses of rf_ShutdownCreate() to not worry about
it ever failing. Shaves another 600 bytes off of an i386 GENERIC kernel.
 1.28  30-Dec-2003  oster No point keeping 'pt' around. It's only ever assigned to.
 1.27  30-Dec-2003  oster rf_FreePDAList() and rf_FreeASMList() wern't using their l_end or
count arguments. Nuke the unused args, and fix up callers and fn
prototypes. Rename l_start arguments.
 1.26  30-Dec-2003  oster Some days you wonder if some of the function declaration consistency
was just an accident in the first place. Cleanup function decls and
a few comments. [ok.. so I wasn't going to fix this many.. but once
you're on a roll....]
 1.25  29-Dec-2003  oster Clean up comments, and generally make more readable. No functional changes.
 1.24  29-Dec-2003  oster - first kick at a major reworking of RAIDframe's memory allocation code:
- all freelists converted to pools
- initialization of structure members in certain cases where
code was relying on specific allocation and usage properties
to keep structures in a "known state" (that doesn't work with
pools!).
- make most pool_get() be "PR_WAITOK" until they can be analyzed
further, and/or have proper error handling added.
- all RF_Mallocs zero the space returned, so there is no difference
between RF_Calloc and RF_Malloc. In fact, all the RF_Calloc()'s
do is tend to do is get things horribly confused.
Make RF_Malloc() the "general memory allocator", with
RF_MallocAndAdd() the "general memory allocator with
allocation list".
- some of these RF_Malloc's et al. are destined to disappear.
- remove rf_rdp_freelist entirely (it's not used anywhere!)
- remove: #include "rf_freelist.h"
- to the files that were relying on the above, add: #include "rf_general.h"
- add: #include "rf_debugMem.h" to rf_shutdown.h to make it happy
about the loss of: #include "rf_freelist.h".

This shrinks an i386 GENERIC kernel by approx 5K. RAIDframe now
weighs in at about 162K on i386.
 1.23  29-Dec-2003  oster [Having received a definite lack of strenuous objection, a small amount
of strenuous agreement, and some general agreement, this commit is
going ahead because it's now starting to block some other changes I
wish to make.]

Remove most of the support for the concept of "rows" from RAIDframe.
While the "row" interface has been exported to the world, RAIDframe
internals have really only supported a single row, even though they
have feigned support of multiple rows.

Nothing changes in configuration land -- config files still need to
specify a single row, etc. All auto-config structures remain fully
forward/backwards compatible.

The only visible difference to the average user should be a
reduction in the size of a GENERIC kernel (i386) by 4.5K. For those
of us trolling through RAIDframe kernel code, a lot of the driver
configuration code has become a LOT easier to read.
 1.22  21-Dec-2003  oster (Although I want this code to Go Away, I might as well fix this bug for completeness)

If we don't have enough memory to allocate the pda freelist, make sure to
cleanup the asm freelist.
 1.21  24-Sep-2002  oster branches: 1.21.6;
Cleanup the comment formatting a bit.
 1.20  23-Sep-2002  oster Ooops.. should have included this 'row' with the last change.
 1.19  23-Sep-2002  oster 'row' was defined and set, but never used. *poof* Thanks to Simon B.
 1.18  21-Sep-2002  oster rf_FreePhysDiskAddr is only needed by paritylogging.
rf_NumFailedDataUnitsInStripe is only needed by PQ, RAID6 and EVENODD.
rf_PrintAccessStripeMap is only needed for RF_DEBUG_MAP.

Thanks Krister!
 1.17  21-Sep-2002  oster rf_FreeAccessStripeMapComponent() goes too. Thanks Krister!
 1.16  21-Sep-2002  oster Nuke rf_AllocAccessStripeMapComponent(). Thanks Krister!
 1.15  19-Sep-2002  oster Introduce RF_DEBUG_MAP, and use it to ignore more rarely used code.
 1.14  14-Sep-2002  oster Everyone and their dog was using RF_ERRORMSG3 to print out the same
sort of error message, over and over again, in different files.
Rather than having the same text repeated in multiple .o files,
create a couple of little functions to do the printing, and save a
bundle of space. Also improves readability of code.
 1.13  03-Aug-2002  oster Nuke two more functions that arn't used anywhere and are only contributing
to kernel bloat.
 1.12  22-May-2002  wiz bcopy -> memcpy. Reviewed by Greg Oster.
 1.11  07-Jan-2002  oster branches: 1.11.8;
Revert previous. Need to be more careful with what gets chopped out.
 1.10  07-Jan-2002  oster Some of this stuff is really just for debugging, so compile it only if
DEBUG is defined.
 1.9  13-Nov-2001  lukem add RCSIDs
 1.8  05-Oct-2001  oster More #if 0's bite the dust.
 1.7  04-Oct-2001  oster Step 2 of the disentanglement. We now look to <dev/raidframe/*> for
the stuff that used to live in rf_types.h, rf_raidframe.h, rf_layout.h,
rf_netbsd.h, rf_raid.h, rf_decluster,h, and a few other places.
Believe it or not, when this is all done, things will be cleaner.

No functional changes to RAIDframe.
 1.6  18-Jul-2001  thorpej branches: 1.6.2;
bzero -> memset
 1.5  29-Jun-2000  oster branches: 1.5.2; 1.5.4;
Nuke a printf that *I* added at least as early as Nov, 1998, and which is
as bogus now as it was then.
 1.4  13-Aug-1999  oster branches: 1.4.2; 1.4.12;
rf_sys.h does not need to be #included in any of these files, and, actually,
is no longer needed at all.
 1.3  05-Feb-1999  oster branches: 1.3.2;
Phase 2 of the RAIDframe cleanup. The source is now closer to KNF
and is much easier to read. No functionality changes.
 1.2  26-Jan-1999  oster RAIDframe cleanup, phase 1. Nuke simulator support, user-land driver,
out-dated comments, and other unneeded stuff. This helps prepare
for cleaning up the rest of the code, and adding new functionality.

No functional changes to the kernel code in this commit.
 1.1  13-Nov-1998  oster RAIDframe, version 1.1, from the Parallel Data Laboratory at
Carnegie Mellon University. Full RAID implementation, including
levels 0, 1, 4, 5, 6, parity logging, and a few other goodies.
Ported to NetBSD by Greg Oster.
 1.3.2.1  29-Jun-2000  he Pull up revision 1.5 (requested by oster):
Remove an incorrect printf.
 1.4.12.1  29-Jun-2000  oster Pullup rev 1.5 (requested by oster)
Nuke a printf that was added at least as early as Nov, 1998, and which is
as bogus now as it was then.
Approved by: thorpej
 1.4.2.1  20-Nov-2000  bouyer Update thorpej_scsipi to -current as of a month ago
A i386 GENERIC kernel compiles without the siop, ahc and bha drivers
(will be updated later). i386 IDE/ATAPI and ncr work, as well as
sparc/esp_sbus. alpha should work as well (untested yet).
siop, ahc and bha will be updated once I've updated the branch to current
-current, as well as machine-dependant code.
 1.5.4.5  10-Oct-2002  jdolecek sync kqueue with -current; this includes merge of gehenna-devsw branch,
merge of i386 MP branch, and part of autoconf rototil work
 1.5.4.4  06-Sep-2002  jdolecek sync kqueue branch with HEAD
 1.5.4.3  23-Jun-2002  jdolecek catch up with -current on kqueue branch
 1.5.4.2  10-Jan-2002  thorpej Sync kqueue branch with -current.
 1.5.4.1  03-Aug-2001  lukem update to -current
 1.5.2.9  18-Oct-2002  nathanw Catch up to -current.
 1.5.2.8  17-Sep-2002  nathanw Catch up to -current.
 1.5.2.7  13-Aug-2002  nathanw Catch up to -current.
 1.5.2.6  20-Jun-2002  nathanw Catch up to -current.
 1.5.2.5  28-Feb-2002  nathanw Catch up to -current.
 1.5.2.4  11-Jan-2002  nathanw More catchup.
 1.5.2.3  14-Nov-2001  nathanw Catch up to -current.
 1.5.2.2  22-Oct-2001  nathanw Catch up to -current.
 1.5.2.1  24-Aug-2001  nathanw Catch up with -current.
 1.6.2.1  11-Oct-2001  fvdl Catch up with -current. Fix some bogons in the sparc64 kbd/ms
attach code. cd18xx conversion provided by mrg.
 1.11.8.2  29-Aug-2002  gehenna catch up with -current.
 1.11.8.1  30-May-2002  gehenna Catch up with -current.
 1.21.6.4  04-Mar-2005  skrll Sync with HEAD.

Hi Perry!
 1.21.6.3  21-Sep-2004  skrll Fix the sync with head I botched.
 1.21.6.2  18-Sep-2004  skrll Sync with HEAD.
 1.21.6.1  03-Aug-2004  skrll Sync with HEAD
 1.37.10.1  19-Mar-2005  yamt sync with head. xen and whitespace. xen part is not finished.
 1.37.8.1  29-Apr-2005  kent sync with -current
 1.38.4.2  03-Sep-2007  yamt sync with head.
 1.38.4.1  21-Jun-2006  yamt sync with head.
 1.39.12.1  24-May-2006  tron Merge 2006-05-24 NetBSD-current into the "peter-altq" branch.
 1.39.10.1  11-May-2006  elad sync with head
 1.39.8.1  24-May-2006  yamt sync with head.
 1.39.6.1  01-Jun-2006  kardel Sync with head.
 1.39.4.1  09-Sep-2006  rpaulo sync with head
 1.40.10.2  10-Dec-2006  yamt sync with head.
 1.40.10.1  22-Oct-2006  yamt sync with head
 1.40.8.1  18-Nov-2006  ad Sync with head.
 1.42.4.1  12-Mar-2007  rmind Sync with HEAD.
 1.43.56.1  13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.43.50.1  28-Apr-2009  skrll Sync with HEAD.
 1.43.40.1  04-May-2009  yamt sync with head.
 1.44.6.1  06-Jun-2011  jruoho Sync with HEAD.
 1.44.4.1  31-May-2011  rmind sync with head
 1.45.14.1  03-Dec-2017  jdolecek update from HEAD
 1.46.4.1  04-Nov-2016  pgoyette Sync with HEAD
 1.46.2.1  05-Dec-2016  skrll Sync with HEAD
 1.47.16.1  10-Jun-2019  christos Sync with HEAD
 1.50.14.1  01-Aug-2021  thorpej Sync with HEAD.

RSS XML Feed