xref: /src/doc/roadmaps/storage

$NetBSD: storage,v 1.9 2012/01/14 22:06:16 agc Exp $

NetBSD Storage Roadmap
======================

This is a small roadmap document, and deals with the storage and file
systems side of the operating system.

The following elements and projects are pencilled in for 6.0, but
please do not rely on them being there.

Features that will be in 6.0:
2. logical volume management
3. a native port of Sun's ZFS
4. ReFUSE, perfuse and pud
6. Support for flash devices - NAND, and flash file system
7. rump extensions
9. in-kernel iSCSI initiator
10. RAIDframe parity map
11. quota system re-work

Features that are planned for future releases:
1. devfs/udevfsd
5. web-based management tools for storage subsystems
8. virtualised disks in userland
12. lfs renovation

We'll continue to update this roadmap as features and dates get firmed up.

Some explanations
=================

1. udevfsd
----------

There has always been discussion over devfs, and experience with it
seems mixed (to be kind). At the same time, carrying around a whole
populated /dev seems quite possible and effective, but maybe a bit
unwieldy. jmcneill's udevfsd addresses this in a different way, and
is currently in othersrc/external/bsd/udevfsd. Not planned for 6.0
right now.

Responsible: jmcneill

2. Logical Volume Management
----------------------------

Based on the Linux lvm2 and devmapper software, with a new kernel component
for NetBSD written. Merged in 5.99.5 sources, will be in 6.0.

Responsible: haad, martin

3. Native port of Sun's ZFS
---------------------------

Two Summer of Code projects have been held, concentrating on the
provision of ZFS support for NetBSD.  Mostly completed by haad, and
building on ver's work, this is the port of Sun's ZFS, with
modifications to make it compile on NetBSD by ad@, and based on the
Sun code for the block layer. Discussions are still taking place to
get the design right for support for the openat(2) system call family,
and the correct architecture for reclaiming vnodes.

The ZFS source code has been committed to the repository.

Responsible: haad, ad, ver

4. ReFUSE, perfuse and pud
--------------------------

FUSE has two interfaces, the normal high-level one, and a lower-level
interface which is closer to the way standard file systems operate. 
manu's perfuse adds the low-level functionality in the same way that
ReFUSE adds the high-level functionality.  In addition, there is the
"pass to userspace device" framework added by pooka as part of rump. 
All 3 will be in 6.0.

Responsible: pooka, manu, agc

5. Web-based Management tools for Storage Subsystems
----------------------------------------------------

Standard tools for managing the storage subsystems that NetBSD
provides, using a standard web-server as the basic user interface on
the storage device, allowing remote management by a standard web
browser.  CIM and related functinoality are interesting datapoints in
this space, although credentials and authentication are always
challenges in this space. Will not make it into 6.0

Responsible: agc

6. Support for flash devices - NAND, and flash file system
----------------------------------------------------------

ahoka has have contributed many things in this area, including
flash(4), flash(9), flashctl(8) and nand(9).  In addition, the
University of Szeged has contributed chfs,
http://en.wikipedia.org/wiki/CHFS, which is described as "the first
open source flash specific file system written for NetBSD".  All of
these will be in 6.0.

Responsible: ahoka

7. RUMP Extensions
------------------

Rump support has been in NetBSD for 2 releases now, and continues to be
developed actively. Recent additions have included cgd support, and smbfs
client support.

Responsible: pooka


8. Virtualised disks in Userland
--------------------------------

For better support of virtualization, a library which provides a consistent 
view of virtualized disk images has been developed by jmcneill. This will
not make it into 6.0, although some extra functionality for reading vmdk
images is available in othersrc/external/bsd/vmdk.

Responsible: jmcneill, agc


9. In-kernel iSCSI Initiator
----------------------------

NetBSD has had a userland implementation of an iSCSI initiator since
NetBSD 4.99.35, based on ReFUSE.  Wasabi Systems kindly contributed their
kernel-based iSCSI initiator, and it will be in 6.0.

Responsible: riz, agc


10. RAIDframe parity map
------------------------

Jed Davis successfully completed a Summer of Code project to implement
parity map zones for RAIDframe.  Parity mapping drastically reduces
the amount of time spent rewriting parity after an unclean shutdown by
keeping better track of which regions might have had outstanding
writes.  Enabled by default; can be disabled on a per-set basis, or
tuned, with the new raidctl(8) commands.

Merged in 5.99.22 sources, and will be in 6.0.  A separate set of
patches is available for NetBSD-5.

Responsible: jld


11. quota system re-work
------------------------

The quota system has been re-worked by bouyer, and is in -current
right now.  dholland is updating and modifying this rework so that it
is a more generalised solution, with better features for security. 
This is expected to be in 6.0, although there is a lot of work to
complete.

Responsible: bouyer, dholland


12. LFS renovation
------------------

LFS had been de-emphasised in the time period leading up to the
5.0 release, but is undergoing some rework by perseant, and dholland
has some contributions in this area too.

Responsible: perseant, dholland


Alistair Crooks, David Holland
Sat Jan 14 05:52:37 PST 2012