Monday, November 9, 2009

ZFS send dedup integrated

Moments ago, one week after zpool dedup was integrated, similar functionality was added for zfs send streams. It looks like OSOL 2010.03 is going to get quite a lot of new ZFS features.

zfs send with the new -D option will dedup the streams created and thereby possibly reducing bandwidth or disk space used by the stream. It's not dependent on pool level dedup.

From PSARC/2009/559:

"OVERVIEW:

"Dedup" is an overall term for technologies that eliminate duplicate
copies of data in storage or memory. This specific application of
dedup is for ZFS send streams, i.e., the output of the 'zfs send' command.
For some kinds of data, much of the content of a send stream consists
of blocks for which identical copies have already been sent earlier
in the stream. This technology replaces later copies of a block with
a reference to the earlier copy. This can significantly reduce the
size of a send stream, which reduces the time it takes to transfer
such a stream over a communication channel."

Here is the changeset: http://hg.genunix.org/onnv-gate.hg/rev/216d8396182e

If all goes well this will together with pool level deup (and lots of other changes) be part of build 128 which should arrive early december.

No comments: