OpenZFS 2.0 version unifies Linux, BSD and adds tons of new features

by



[ad_1]

A stylized illustration of tiny computer components.
Enlarge / OpenZFS 2.0.0 brings a ton of new features and performance improvements to Linux and BSD platforms.

Aurich Lawson

On Monday, ZFS lead developer on Linux, Brian Behlendorf, released OpenZFS 2.0.0 on GitHub. In addition to many new features, the announcement ends the old distinction between “ZFS on Linux” and ZFS elsewhere (eg, on FreeBSD). This decision has been a long time coming – the FreeBSD community presented their version of the roadmap two years ago – but this is the version that makes it official.

Availability

The new version of OpenZFS 2.0.0 is already available on FreeBSD, where it can be installed from ports (replacing the base ZFS system) on FreeBSD 12 systems and will be the base version of FreeBSD in the upcoming FreeBSD 13 Under Linux the situation is a bit more uncertain and largely depends on the Linux distribution in play.

Users of Linux distributions that use OpenZFS kernel modules built by DKMS will tend to get the new version quite quickly. Better-supported but slower Ubuntu users likely won’t see OpenZFS 2.0.0 until Ubuntu 21.10, almost a year from now. For Ubuntu users who want to live on the edge, the popular but third-party, individually managed jonathonf PPA could make it available much sooner.

OpenZFS 2.0.0 modules can be built from source for Linux kernels 3.10-5.9 – but most users should stick to getting prebuilt modules from established distributions or developers. “Way beyond the beaten track” is not a phrase that one should generally apply to the file system that contains its precious data!

New features

  • Sequential Resilver – rebuilding degraded arrays in ZFS has always been very different from conventional RAID. On nearly empty arrays, ZFS rebuild – known as “resilvering” – was much faster because ZFS only needed to hit the used part of the disk rather than cloning each sector to the entire drive. But this process involved an abundance of random I / O. Thus, on nearly full bays, the block-by-block reconstruction of the entire drive of conventional RAID, more pedestrian, proceeded much faster. With sequential resilvering, ZFS gets the best of both worlds: largely sequential access while ignoring unused portions of the drive (s) involved.
  • Persistent L2ARC – One of the coolest features of ZFS is its advanced read cache, known as ARC. Systems with very large and hot working sets can also implement an SSD-based read cache called L2ARC, which fills up from blocks in ARC near eviction. Historically, one of the biggest issues with L2ARC is that while the underlying SSD is persistent, the L2ARC itself is not – it goes blank on every reboot (or export and import from the pool). This new feature allows L2ARC data to remain available and viable between pool import / export cycles (including system restarts), greatly increasing the potential value of the L2ARC device.
  • Zstd Compression Algorithm – OpenZFS offers transparent, controllable in-line compression with per-dataset granularity. Traditionally, the most commonly used algorithm is lz4, a streaming algorithm offering a relatively low compression ratio but very light CPU load. OpenZFS 2.0.0 supports zstd – an algorithm designed by Yann Collet (the author of lz4) which aims to provide compression similar to gzip, with CPU load similar to lz4.

    • The x axis is the transfer speed, the y axis is the compression rate. Look for ZSTD1-19 on the top left in dark blue, ZSTD-FAST on the bottom right in light blue, and LZ4 in a single orange dot to the right of ZSTD-FAST.

    • Zstd is the dark blue cluster at the top left, zstd-fast is the light blue line that crosses the bottom of the graph, and LZ4 is the only orange point a little to the right of the vertical segment of zstd-fast.

    These charts are a little hard to follow, but essentially they show that zstd is achieving its goals. During compression (writes to disk), zstd-2 is more efficient than even gzip-9 while maintaining high throughput.

    Compared to lz4, zstd-2 achieves 50% greater compression in exchange for a 30% throughput penalty. On the decompression side (playing the disc), the rate penalty is slightly higher, at around 36%.

    Keep in mind that the throughput “penalties” described assume a negligible bottleneck on the storage medium itself. In practice, most processors can run rings around most storage media (even relatively slow processors and fast SSDs). ZFS users are generally used to seeing lz4 compression accelerate real world workloads, don’t slow them down!

  • Redacted line – this one is a bit of a puzzle. Suppose some parts of your data born want to back up using ZFS Replication. First, you clone the dataset. Then you delete the sensitive data from the clone. Then, you bookmark the parent dataset, which marks the blocks that have moved from parent to clone. Finally, you can send the parent dataset to its backup target, including the --redact redaction_bookmark argument – and this reproduces the noblocks sensitive only to the backup target.

Additional improvements and changes

In addition to the main features described above, OpenZFS 2.0.0 brings fallocate support; improved and reorganized man pages; superior performance for zfs destroy, zfs send, and zfs receive;more efficient memory management; and optimized encryption performance. During this time, some rarely used features (deduplicated send streams, dedupditto blocks, and zfs_vdev_scheduler module option) have all been deprecated.

For a full list of changes, please see the original release announcement on GitHub at https://github.com/openzfs/zfs/releases/tag/zfs-2.0.0.

[ad_2]

Source link