Go to file
Alexander Motin 7742e29387 Optimize RAIDZ expansion
- Instead of copying one ashift-sized block per ZIO, copy as much
as we have contiguous data up to 16MB per old vdev.  To avoid data
moves use gang ABDs, so that read ZIOs can directly fill buffers
for write ZIOs.  ABDs have much smaller overhead than ZIOs in both
memory usage and processing time, plus big I/Os do not depend on
I/O aggregation and scheduling to reach decent performance on HDDs.
 - Reduce raidz_expand_max_copy_bytes to 16MB on 32bit platforms.
 - Use 32bit range tree when possible (practically always now) to
slightly reduce memory usage.
 - Use ZIO_PRIORITY_REMOVAL for early stages of expansion, same as
for main ones.
 - Fix rate overflows in `zpool status` reporting.

With these changes expanding RAIDZ1 from 4 to 5 children I am able
to reach 6-12GB/s rate on SSDs and ~500MB/s on HDDs, both are
limited by devices instead of CPU.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by:	Alexander Motin <mav@FreeBSD.org>
Sponsored by:	iXsystems, Inc.
Closes #15680
Closes #16819
2024-12-06 09:05:02 -08:00
.github ZTS: Add Fedora 41, remove Fedora 39 2024-11-01 10:03:05 -07:00
cmd Optimize RAIDZ expansion 2024-12-06 09:05:02 -08:00
config Linux: Fix detection of register_sysctl_sz 2024-12-02 18:14:26 -08:00
contrib Fix inconsistent mount options for ZFS root 2024-10-21 13:02:07 -07:00
etc etc/init.d: decide which variant to use at build time. 2024-04-08 16:52:24 -07:00
include Allow dsl_deadlist_open() return errors 2024-12-05 09:33:21 -08:00
lib Add ability to scrub from last scrubbed txg 2024-12-05 09:33:21 -08:00
man Add ability to scrub from last scrubbed txg 2024-12-05 09:33:21 -08:00
module Optimize RAIDZ expansion 2024-12-06 09:05:02 -08:00
rpm fix: block incompatible kernel from being installed 2024-11-21 08:24:37 -08:00
scripts config: remove ZFS_GLOBAL_ZONE_PAGE_STATE and ZFS_ENUM_* generation 2024-09-18 11:23:50 -07:00
tests Fix false assertion in dmu_tx_dirty_buf() on cloning 2024-12-05 11:49:06 -08:00
udev vdev_id: multi-lun disks & slot num zero pad 2024-10-09 13:44:55 -07:00
.cirrus.yml CI: add FreeBSD build with Cirrus CI 2023-10-06 08:50:26 -07:00
.editorconfig Add an .editorconfig; document git whitespace settings 2020-01-27 13:32:52 -08:00
.gitignore Packaging: Auto-generate changelog during configure (#15528) 2023-11-16 08:58:47 -08:00
.gitmodules .gitmodules: link to openzfs github repository 2021-04-12 09:37:23 -07:00
.mailmap AUTHORS: refresh with recent new contributors 2024-11-15 15:08:59 -08:00
AUTHORS AUTHORS: refresh with recent new contributors 2024-11-15 15:08:59 -08:00
autogen.sh Ubuntu 22.04 integration: ShellCheck 2022-11-18 11:24:48 -08:00
CODE_OF_CONDUCT.md Documentation corrections 2022-12-22 11:34:28 -08:00
configure.ac config/kernel: enforce maximum kernel version, with escape hatch 2024-09-23 10:44:49 -07:00
copy-builtin copy-builtin: add hooks with sed/>> 2022-05-10 10:17:43 -07:00
COPYRIGHT Fix typos 2020-06-09 21:24:09 -07:00
LICENSE Update build system and packaging 2018-05-29 16:00:33 -07:00
Makefile.am Process script directory for all configs 2022-10-27 16:45:14 -07:00
META Linux 6.12 compat: META 2024-11-21 08:24:37 -08:00
NEWS Fix NEWS file 2020-08-26 21:44:41 -07:00
NOTICE Update build system and packaging 2018-05-29 16:00:33 -07:00
README.md FreeBSD: remove support for FreeBSD < 13.0-RELEASE (#16372) 2024-08-05 16:56:45 -07:00
RELEASES.md Add RELEASES.md file 2021-04-02 16:33:40 -07:00
TEST Remove CI builder customization from TEST 2020-03-16 10:46:03 -07:00
zfs.release.in Move zfs.release generation to configure step 2012-07-12 12:22:51 -07:00

img

OpenZFS is an advanced file system and volume manager which was originally developed for Solaris and is now maintained by the OpenZFS community. This repository contains the code for running OpenZFS on Linux and FreeBSD.

codecov coverity

Official Resources

Installation

Full documentation for installing OpenZFS on your favorite operating system can be found at the Getting Started Page.

Contribute & Develop

We have a separate document with contribution guidelines.

We have a Code of Conduct.

Release

OpenZFS is released under a CDDL license. For more details see the NOTICE, LICENSE and COPYRIGHT files; UCRL-CODE-235197

Supported Kernels

  • The META file contains the officially recognized supported Linux kernel versions.
  • Supported FreeBSD versions are any supported branches and releases starting from 13.0-RELEASE.