Go to file
Rob Norris 46c4f2ce0b
dsl_dataset: put IO-inducing frees on the pool deadlist
dsl_free() calls zio_free() to free the block. For most blocks, this
simply calls metaslab_free() without doing any IO or putting anything on
the IO pipeline.

Some blocks however require additional IO to free. This at least
includes gang, dedup and cloned blocks. For those, zio_free() will issue
a ZIO_TYPE_FREE IO and return.

If a huge number of blocks are being freed all at once, it's possible
for dsl_dataset_block_kill() to be called millions of time on a single
transaction (eg a 2T object of 128K blocks is 16M blocks). If those are
all IO-inducing frees, that then becomes 16M FREE IOs placed on the
pipeline. At time of writing, a zio_t is 1280 bytes, so for just one 2T
object that requires a 20G allocation of resident memory from the
zio_cache. If that can't be satisfied by the kernel, an out-of-memory
condition is raised.

This would be better handled by improving the cases that the
dmu_tx_assign() throttle will handle, or by reducing the overheads
required by the IO pipeline, or with a better central facility for
freeing blocks.

For now, we simply check for the cases that would cause zio_free() to
create a FREE IO, and instead put the block on the pool's freelist. This
is the same place that blocks from destroyed datasets go, and the async
destroy machinery will automatically see them and trickle them out as
normal.

Sponsored-by: Klara, Inc.
Sponsored-by: Wasabi Technology, Inc.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Signed-off-by: Rob Norris <rob.norris@klarasystems.com>
Closes #6783
Closes #16708
Closes #16722 
Closes #16697
2024-11-13 07:38:42 -08:00
.github CI: Automate some GitHub PR status labels manipulations 2024-11-04 17:16:32 -08:00
cmd JSON: fix user properties output for zpool list 2024-11-11 09:46:19 -08:00
config Use simple folio migration function 2024-11-06 11:44:10 -08:00
contrib Fix inconsistent mount options for ZFS root 2024-10-17 09:09:39 -04:00
etc etc/init.d: decide which variant to use at build time. 2024-04-08 16:52:24 -07:00
include module: unicode: remove unused tolower transformations 2024-11-04 17:26:35 -08:00
lib Fix user properties output for zpool list 2024-11-11 09:46:45 -08:00
man zfs(4): remove "experimental" from zfs_bclone_enabled 2024-11-01 14:43:25 -07:00
module dsl_dataset: put IO-inducing frees on the pool deadlist 2024-11-13 07:38:42 -08:00
rpm Bump SONAME of libzfs and libzpool 2024-10-06 14:49:33 -07:00
scripts config: remove ZFS_GLOBAL_ZONE_PAGE_STATE and ZFS_ENUM_* generation 2024-09-18 11:23:50 -07:00
tests Use <fcntl.h> instead of <sys/fcntl.h> 2024-11-07 11:20:37 -08:00
udev vdev_id: multi-lun disks & slot num zero pad 2024-10-08 17:43:04 -07:00
.cirrus.yml CI: add FreeBSD build with Cirrus CI 2023-10-06 08:50:26 -07:00
.editorconfig Add an .editorconfig; document git whitespace settings 2020-01-27 13:32:52 -08:00
.gitignore Packaging: Auto-generate changelog during configure (#15528) 2023-11-16 08:58:47 -08:00
.gitmodules .gitmodules: link to openzfs github repository 2021-04-12 09:37:23 -07:00
.mailmap AUTHORS: refresh with recent new contributors 2024-09-24 09:03:05 -07:00
AUTHORS AUTHORS: refresh with recent new contributors 2024-09-24 09:03:05 -07:00
autogen.sh Ubuntu 22.04 integration: ShellCheck 2022-11-18 11:24:48 -08:00
CODE_OF_CONDUCT.md Documentation corrections 2022-12-22 11:34:28 -08:00
configure.ac config/kernel: enforce maximum kernel version, with escape hatch 2024-09-23 10:44:49 -07:00
copy-builtin copy-builtin: add hooks with sed/>> 2022-05-10 10:17:43 -07:00
COPYRIGHT Fix typos 2020-06-09 21:24:09 -07:00
LICENSE Update build system and packaging 2018-05-29 16:00:33 -07:00
Makefile.am Process script directory for all configs 2022-10-27 16:45:14 -07:00
META Update META 2024-10-04 14:20:10 -07:00
NEWS Fix NEWS file 2020-08-26 21:44:41 -07:00
NOTICE Update build system and packaging 2018-05-29 16:00:33 -07:00
README.md FreeBSD: remove support for FreeBSD < 13.0-RELEASE (#16372) 2024-08-05 16:56:45 -07:00
RELEASES.md Add RELEASES.md file 2021-04-02 16:33:40 -07:00
TEST Remove CI builder customization from TEST 2020-03-16 10:46:03 -07:00
zfs.release.in Move zfs.release generation to configure step 2012-07-12 12:22:51 -07:00

img

OpenZFS is an advanced file system and volume manager which was originally developed for Solaris and is now maintained by the OpenZFS community. This repository contains the code for running OpenZFS on Linux and FreeBSD.

codecov coverity

Official Resources

Installation

Full documentation for installing OpenZFS on your favorite operating system can be found at the Getting Started Page.

Contribute & Develop

We have a separate document with contribution guidelines.

We have a Code of Conduct.

Release

OpenZFS is released under a CDDL license. For more details see the NOTICE, LICENSE and COPYRIGHT files; UCRL-CODE-235197

Supported Kernels

  • The META file contains the officially recognized supported Linux kernel versions.
  • Supported FreeBSD versions are any supported branches and releases starting from 13.0-RELEASE.