Both Mike and Brendan quits Oracle

The last of the tree DTrace creators are now leaving Oracle, Mike Shapiro, who also was part of the FishWorks core team. Another member of the FishWorks team was Brendan Greg who also resigned from Oracle. Those of you who didn't now about Bredan before probably got to know who he was when he became famous for screaming at disks in a online video.

DTrace and ZFS is the core foundation of the S7000 storage appliance, now the whole team who invented DTrace is gone and so are the two core persons behind ZFS, Jeff Bonwick and Bill Moore.

There are a lot of other great people behind these products, but the ones listed above are rare and extremly talented engineers and have all been part of breaking new ground in their area of expertice. The FishWorks team even created a usable web interface, a rare thing comming from Sun.

Mike Shapiro: End of file
Brendan Gregg: G'Day and Goodbye
Brendan Gregg screaming, Unusual disk latency
FishWorks, Now it can be told

Nexenta OpenStorage summit

Next week Nexenta will be hosting it's first annual OpenStorage summit in Palo Alto, CA. Besides obvious speakers such as Evan Powell, CEO of Nexenta several people from the ZFS/OpenSolaris community will also be speaking:

Bill Moore, Ex-Sun ZFS Co-creator
Jeff Bonwick, Ex-Sun creator of ZFS
Ben Rockwood, director of systems at Joyent.

Nexenta is as you probably know an OpenStorage vendor which builds it's storage appliance on top of ZFS. They are probably part of the reason Oracle cut the cord on the OpenSolaris source since their product competed with Oracles own OpenStorage appliances, which also are built on ZFS. Nexenta plans to continue to develop the ZFS/OpenSolais source and are sponsors of the illumos project which is the de-facto development gate of the OpenSolaris source now.

It will be interesting to see how they plan to handle the fork of ZFS now that Oracle have continued their development internally, ZFS have gained bug fixes but more importantly new features in newer versions of the on-disk format. If Nexenta plans to enhance ZFS to a degree that requires new version of the pool the two implementations will no longer be compatible.

Nexenta will fall behind Oracle Solaris/Oracle OpenStorage in terms of ZFS features, at least for a while. Oracle has plenty of source already created for ZFS which is yet to be integrated in their Solaris Next source, one of them is the BP rewrite project which has been in the making for years, another is ZFS on-disk encryption which has already been integrated into the internal Solaris gate at Oracle

Will Nexenta sit out and hope for the ZFS source to be released after Solaris 11 (late next year at the earliest) or will the make larger enhancements by their own. Both scenarios are possible while i doubt Nexenta will sit and await for Oracle. The ZFS code already released is stable and contains features not yet integrated into Nexenta products. New development is also possible, especially since several key persons behind ZFS have left Oracle and there are other developers, especially with a company found the development.

In the end this will affect all users of ZFS which are not Oracle customers, Nexenta and illumos are probably where future free ZFS development will be done. All users who want's a free OS with the latest possible ZFS bits are going to be using Nexenta Core or OpenIndiana or some other distribution based on the illumos source.

Nexenta announces first summit
Nexenta announces final agenda and sponsors for its first annual OpenStorage summit

Zones should be able to be NFS servers

NFS service in local zones have been frequently requested since the introduction of zones in Solaris 10, the change request for this even predates Solaris 10 by two years, it was filled in 2003. I've earlier mentioned that some works was being done to this including PSARC/2010/208. Now it seems that this has been implemented and will be integrated into internal build 154 of Solaris Next. This is hopefully in time for the first release of Solaris 11 Express which is expected next month.

Update: I was a bit quick on this one, build 154 should close in the end of next month, that does not work very well with a November release of Solaris Express, so it might be delayed or more probably it will not be part of the first Express release.

Zones should be able to be NFS servers, bugid 4964859
NFS Instances with NFS shares.
PSARC/2010/208

Solaris 11 Express 2010.11 and ZFS

Besides ZFS data set encryption the upcoming Solaris Express release (Which seems to be tagged 2010.11) will contain several enhancements to ZFS besides dataset encryption that was mentioned earlier:

• RAID-Z/mirror hybrid allocator
"Normally mirror vdevs contain mirrored data, RAID-Z vdevs contain RAID-Z data, etc. However, for latency-sensitive metadata, we can use a mirrored layout across the children of a RAID-Z vdev. This ensures that such metadata can be read in a single I/O."


• Support for decompression, checksumming and raidz in zdb -R will make it much easier to debug ZFS errors on disk or just to learn more about the on-disk format of ZFS.


• Performance improvements for listing ZFS filesystems.

The downside of this is that these are features only present in Oracles internal gate and not other distribution besides Solaris 11/Express and the S7000/FishWork appliances can use them until and if Oracle releases the source after Solaris 11 is released.

I hope the Solaris 11 Express license will allow me to use Solaris 11 for my private storage needs, S7000 are nice and deployed at work but still a bit too expensive for my datacenter at home ;)

6977913 RAID-Z/mirror hybrid allocator
6757444 want zdb -R to support decompression, checksumming and raid-z

ZFS crypto integrated

Support for encrypted ZFS datasets have been integrated into Solaris Next Development, build snv_148. This means that there is a good chance it will be available with the first release of Solaris 11 Express which is expected later this year.

This feature have been in the works for years with the original feature request filed in 2003. This will be a welcome addition for ZFS users with sensitive data and/or laptops.

After this only one major component is missing from ZFS, the block pointer rewrite feature. It will make it possible to encrypt/compress/deducplicate existing data in the pools as well as shrink pools and defragment them without the need to move the data out of the pool and in again.

Update:
As noted I failed to include a reference, here is bugid 4854202.

ZFS on disk encryption support
Darren in ZFS crypto video interview
ZFS crypto pushed to next year