local ZFS: improve memory-limit section

recent forum threads showed that its not clear to all that they need
to reboot to activat modprobe.d parameters, or that they can also
directly write to /sys for a non-presistent but live change

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

local-zfs.adoc

@@ -450,25 +450,39 @@ other settings are optional.
 Limit ZFS Memory Usage
 ~~~~~~~~~~~~~~~~~~~~~~
 
-It is good to use at most 50 percent (which is the default) of the
-system memory for ZFS ARC to prevent performance shortage of the
-host. Use your preferred editor to change the configuration in
-`/etc/modprobe.d/zfs.conf` and insert:
+ZFS uses '50 %' of the host memory for the **A**daptive **R**eplacement
+**C**ache (ARC) by default. Allocating enough memory for the ARC is crucial for
+IO performance, so reduce it with caution. As a general rule of thumb, allocate
+at least +2 GiB Base + 1 GiB/TiB-Storage+. For example, if you have a pool with
++8 TiB+ of available storage space then you should use +10 GiB+ of memory for
+the ARC.
+
+You can change the ARC usage limit for the current boot (a reboot resets this
+change again) by writing to the +zfs_arc_max+ module parameter directly:
+
+----
+ echo "$[10 * 1024*1024*1024]" >/sys/module/zfs/parameters/zfs_arc_max
+----
+
+To *permanently change* the ARC limits, add the following line to
+`/etc/modprobe.d/zfs.conf`:
 
 --------
 options zfs zfs_arc_max=8589934592
 --------
 
-This example setting limits the usage to 8GB.
+This example setting limits the usage to 8 GiB ('8 * 2^30^').
 
 [IMPORTANT]
 ====
-If your root file system is ZFS you must update your initramfs every
+If your root file system is ZFS, you must update your initramfs every
 time this value changes:
 
 ----
 # update-initramfs -u
 ----
+
+You *must reboot* to activate these changes.
 ====