mirror_zfs/cmd
Alexander Motin a44eaf1690
Optimize RAIDZ expansion
- Instead of copying one ashift-sized block per ZIO, copy as much
as we have contiguous data up to 16MB per old vdev.  To avoid data
moves use gang ABDs, so that read ZIOs can directly fill buffers
for write ZIOs.  ABDs have much smaller overhead than ZIOs in both
memory usage and processing time, plus big I/Os do not depend on
I/O aggregation and scheduling to reach decent performance on HDDs.
 - Reduce raidz_expand_max_copy_bytes to 16MB on 32bit platforms.
 - Use 32bit range tree when possible (practically always now) to
slightly reduce memory usage.
 - Use ZIO_PRIORITY_REMOVAL for early stages of expansion, same as
for main ones.
 - Fix rate overflows in `zpool status` reporting.

With these changes expanding RAIDZ1 from 4 to 5 children I am able
to reach 6-12GB/s rate on SSDs and ~500MB/s on HDDs, both are
limited by devices instead of CPU.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by:	Alexander Motin <mav@FreeBSD.org>
Sponsored by:	iXsystems, Inc.
Closes #15680
Closes #16819
2024-12-06 08:50:16 -08:00
..
raidz_test build: rename FORCEDEBUG_CPPFLAGS to LIBZPOOL_CPPFLAGS 2024-08-27 12:53:27 -07:00
zdb zdb_il: use flex array member to access ZIL records 2024-12-04 19:03:20 -05:00
zed zed: prevent automatic replacement of offline vdevs 2024-11-15 15:08:16 -08:00
zfs JSON: fix user properties output for zfs list 2024-11-07 11:31:14 -08:00
zinject Parallel pool import 2024-04-22 09:42:38 -07:00
zpool Optimize RAIDZ expansion 2024-12-06 08:50:16 -08:00
zpool_influxdb Do not report bytes skipped by scan as issued. 2023-06-30 08:47:13 -07:00
zstream zio_compress: introduce max size threshold 2024-09-19 17:23:58 -07:00
arc_summary Remove hash_elements_max accounting from DBUF and ARC 2024-11-19 07:00:16 -08:00
arcstat.in arcstat: add structural, types, states breakdown 2024-09-18 11:44:18 -07:00
dbufstat.in Consider dnode_t allocations in dbuf cache size accounting 2023-11-17 13:25:53 -08:00
fsck.zfs.in cmd: move single-file binaries up, extract udev programs to udev/ 2022-05-10 10:20:34 -07:00
Makefile.am build: rename FORCEDEBUG_CPPFLAGS to LIBZPOOL_CPPFLAGS 2024-08-27 12:53:27 -07:00
mount_zfs.c Make mount.zfs(8) calling zfs_mount_at for legacy mounts as well 2024-08-23 10:39:09 -07:00
zfs_ids_to_path.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
zgenhostid.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
zhack.c Support for longnames for files/directories (Linux part) 2024-10-01 13:40:27 -07:00
zilstat.in zil: add stats for commit failure/fallback (#16315) 2024-07-25 16:53:59 -07:00
ztest.c ztest: Fix scrub check in ztest_raidz_expand_check() 2024-10-08 20:41:17 -07:00
zvol_wait zvol_wait logic may terminate prematurely 2022-10-11 12:12:04 -07:00