Commit Graph

2025 Commits

Author SHA1 Message Date
jiangcuo
1466284ce8
Merge branch 'proxmox:master' into master 2025-03-09 16:48:50 +08:00
a9c0a93f12 update pve-storage to 8.3.4-2 2025-03-09 16:47:27 +08:00
2d73cfab28 add pvebcache cli 2025-03-09 16:47:10 +08:00
a7f1798e51 bump libpve-storage-perl to 8.3.4-1 2025-02-26 17:24:27 +08:00
955737a092 bump libpve-storage-perl to 8.3.4 2025-02-26 17:17:29 +08:00
Fiona Ebner
9dd136c8df btrfs: volume export: add note to clarify special handling of 'snapshot' argument
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-02-24 10:18:38 +01:00
Fiona Ebner
424280a16c btrfs: avoid superfluous check in foreach_snapshot_of_subvol() helper
The helper iterates with the BTRFS_SNAPSHOT_REGEX regular expression,
so there will always be a snapshot name.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2025-02-24 10:18:35 +01:00
jiangcuo
fd62aa1ed9 * Add vdisk_clone_pxvirt func. This func will force create link clone for pxvditemplate vm.
* enable snapshot link clone on zfspool.
2025-02-23 16:21:43 +08:00
jiangcuo
165bbf5944 bump pve-storage to 8.3.3+port1 2025-02-22 14:19:18 +08:00
jiangcuo
50433e9e62
Merge branch 'proxmox:master' into master 2025-02-21 19:31:02 +08:00
jiangcuo
97b959797f Qcow2 can't use clonedisk fn 2025-02-21 11:07:16 +08:00
Maximiliano Sandoval
a25b1050ce btrfs: rename snapshot parameter
It was originally introduced as $snapshot to match the argument name of
foreach_subvol, we rename it here to make it clear that it only contains
the snapshot name.

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2025-02-20 15:57:02 +01:00
Maximiliano Sandoval
3abd10cedf btrfs: remove foreach_subvol
This method is not used anymore.

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2025-02-20 15:57:02 +01:00
Maximiliano Sandoval
cae30fef56 btrfs: rename volume regex
The regex only lists snapshots.

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2025-02-20 15:57:02 +01:00
Maximiliano Sandoval
d3b84d73fd btrfs: forcefully set image to readwrite
When a subvolume is transferred via btrfs send/receive the resulting
image contains the received_uuid property set. This property is required
to do incremental snapshots.

A downside though is that once the received_uuid property is set, it is
not possible to make the image readwrite again without the force (-f)
flag, and in such case the received_uuid property is lost. Since we know
the images are only set to rw for the duration of the move, it is safe
to set the flag forcefully and then in a future commit add the
received_uuid property by force.

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2025-02-20 15:57:02 +01:00
Maximiliano Sandoval
c2db0cb7d5 btrfs: use foreach_snapshot_of_subvol helper in free_image
Replaces the current use without changes. The `$dir` variable is not
used anymore at that moment so it is defined later.

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2025-02-20 15:57:02 +01:00
Maximiliano Sandoval
c05ef0dbce fix #3873: btrfs: use foreach_snapshot_of_subvol helper in volume_export
Suppose we are taking a snapshot of VM 100's disk-0. The
dir_glob_foreach runs over $path=/subvolume/images/100, lists all
snapshot names and appends their names to the path of the disk, e.g.
/subvolume/images/vm-100-disk-0@SNAP_NAME, but the original directory
$path might contain a second disk `vm-100-disk-1` which is also listed
by the dir_glib_foreach.

By using the helper we only iterate over the snapshots of the guest.

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2025-02-20 15:57:02 +01:00
Maximiliano Sandoval
171b760d81 btrfs: add helper to iterate over snapshots of a subvolume
In this context a subvolume means a BTRFS subvolume.
`$volume\@$snap_name` would be for example
`btrfs_volume/images/102/vm-102-disk-0@snap_name`.

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2025-02-20 15:57:02 +01:00
jiangcuo
ccf250acf7 Add clone_image_pxvirt for pxvirt 2025-02-20 16:11:55 +08:00
Daniel Kral
70955c2d34 api: {upload, download}_url: factor out common parameter hash accesses
Minor cleanup to reduce the amount of `$param->{...}` to variables in
the upload and download url API handler.

Signed-off-by: Daniel Kral <d.kral@proxmox.com>
2025-02-19 16:25:40 +01:00
Dmitry Petrov
bdabd1296b iscsidirect plugin: do not use cache in list_images()
A static cache key 'directiscsi' was used to cache storeid-specific
information. This was causing issues in case of multiple instances.

Drop $cache usage instead of fixing the issue as there is no caller
using it and the same portal/storeid multiple times.

Remove $storeid from iscsi_ls() and its callers.

Signed-off-by: Dmitry Petrov <dpetrov67@gmail.com>
2025-02-12 14:28:34 +01:00
Dmitry Petrov
9b7c1746f1 rbd plugin: do not use cache in list_images()
A static cache key 'rbd' was used to cache pool/storeid-specific
information. This was causing issues in case of multiple RBD pools.

Drop $cache usage instead of fixing the issue as there is no caller
using it and the same pool/storeid multiple times.

Remove $pool from rbd_ls() and get_rbd_path($scfg) from list_images().

Signed-off-by: Dmitry Petrov <dpetrov67@gmail.com>
2025-02-12 14:28:32 +01:00
jiangcuo
6c540ad496 Add bcache support 2025-02-08 14:42:51 +08:00
jiangcuo
e91ecf2aeb
Merge branch 'proxmox:master' into master 2025-02-08 14:23:55 +08:00
Fabian Grünbichler
e5f4af47d0 bump version to 8.3.3
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-12-19 15:16:45 +01:00
Fiona Ebner
b3c844b858 rbd plugin: list: drop outdated error message check
This became outdated after Ceph commit ac547a5b7dc ("rbd: return 0 and
an empty list when pool is entirely empty") 11 years ago. See also:
https://tracker.ceph.com/issues/6693

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2024-12-19 12:38:59 +01:00
Fiona Ebner
a1140d77d0 plugins: volume import: align size up to 1KiB
Previously, the size was rounded down which, in case of an image with
non-1KiB-aligned sze (only possible for external plugins or manually
created images) would lead to errors when attempting to write beyond
the end of the too small allocated target image.

For image allocation, the size is already rounded up to the
granularity of the storage. Do the same for import.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2024-12-19 12:38:59 +01:00
Max Carrara
c41d7755c3 common: introduce common module
This module's purpose is to provide shared functions, constants, etc.
for storage plugins and storage-related operations.

It starts out with a align_size_up() function, that will (initially)
be used for volume import.

[FE: start out with a different function for my use case
     fixup Makefile]

Originally-by: Max Carrara <m.carrara@proxmox.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2024-12-19 12:38:59 +01:00
Fiona Ebner
cc0efbd250 iscsi direct plugin: support volume export
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Tested-by: Filip Schauer <f.schauer@proxmox.com>
2024-12-19 12:38:08 +01:00
Fiona Ebner
d8a67cb6bf iscsi plugin: support volume export
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Tested-by: Filip Schauer <f.schauer@proxmox.com>
2024-12-19 12:37:58 +01:00
Fiona Ebner
e2cf3cea96 rbd plugin: improve volume exists helper
Currently, the helper would not distinguish between different kinds
of errors. Instead of relying on an error, list the images and check
there.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2024-12-19 12:34:14 +01:00
Fiona Ebner
b6f4b89640 rbd plugin: implement volume import/export
For now, only 'raw+size' is supported and it's not possible to
export/import with snapshots. The volume or snapshot is exported or
imported via the corresponding 'rbd' commands.

Introducing an 'rbd' transport format might be feasible for more
complete (i.e. with snapshots, incremental) transfer between two RBD
storages.

Use the '--dest-pool' switch rather than '-p' for import, because the
latter is deprecated.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2024-12-19 12:34:14 +01:00
Fiona Ebner
3bf2193b44 rbd plugin: factor out helper to check if volume already exists
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2024-12-19 12:34:14 +01:00
Fiona Ebner
9adc4c69ef export: redirect stdout to avoid any unrelated messages ending up in the export stream
Current export implementations luckily seems to not run into this
issue yet. However, for the upcoming implementation for RBD, mapping a
volume would print the device path to STDOUT, thus messing up the
export stream.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2024-12-19 12:34:14 +01:00
Fiona Ebner
7166db2ce4 rbd plugin: schema: document default value for 'krbd' setting
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2024-12-19 12:34:14 +01:00
Fiona Ebner
a16bf6f791 plugin: export/import: fix calls to path() method
The plugin API does not require call context detection for the
returned value of the path() method. See other plugins like
ISCSIDirect/ZFS that do not implement it. So do not expect it for
external plugins either.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2024-12-19 12:34:14 +01:00
Thomas Lamprecht
edb18d6386 bump version to 8.3.2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-12-11 15:41:54 +01:00
Fabian Grünbichler
652154c0b0 btrfs: fix handling of non-images volumes
previously, only 'images' returned a format, now other volume types do too, so
differentiate where needed.

reported on the forum: https://forum.proxmox.com/threads/158888/

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-12-11 14:53:07 +01:00
Fabian Grünbichler
c0304618f5 bump version to 8.3.1
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-12-10 14:33:11 +01:00
Fabian Grünbichler
a9e0f50d4a file_size_info: add warning when falling back to raw format
in case this gets called with an explicit format that is none of:
- 'auto-detect'
- 'subvol'
- a member of the list of known "qemu" formats

this should only affect third-party storage plugins that either call this
directly with a format, or via inherited code that gets a format from
parse_volname and passes it to file_size_info.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
2024-12-10 14:31:50 +01:00
Fabian Grünbichler
081ce2c9df esxi: fix return value of volume_size_info for vmx volumes
in case of an array context, it should also return the format, else a caller
might assume it failed.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
2024-12-10 14:31:49 +01:00
Fabian Grünbichler
bfb3dff319 storage: plugin: return 'raw' format when parsing non-image volumes
since `volume_size_info` passes the parsed format to `file_size_info`, which
prints a warning if the format is undef before falling back to auto-detection,
and these should always be treated as raw files anyway.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
2024-12-10 14:31:46 +01:00
Shannon Sterz
93c1af6ba4 esxi: anchor vmdk regex in parse_volname
otherwise, `volname.vmdk.raw` would be picked up as a `vmdk` formatted
image, which may not be desired.

Signed-off-by: Shannon Sterz <s.sterz@proxmox.com>
2024-12-10 10:21:21 +01:00
Fabian Grünbichler
5784c84e8f bump version to 8.3.0
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-12-09 12:01:23 +01:00
Fabian Grünbichler
f1ea2c5c89 file_size_info: add missing format parameters
these calls would print spurious warnings otherwise.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-12-09 11:48:01 +01:00
Fabian Grünbichler
a8d246e784 bump version to 8.2.10
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-12-09 10:42:44 +01:00
Fiona Ebner
0cc5492707 esxi: parse volname: explicitly return format for vmx configuration file
For consistency with the list_images() call and so that callers can
better deal with it.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2024-12-09 08:55:10 +01:00
Fiona Ebner
541064a95b list images: return 'invalid' format value for image that cannot be queried in expected format
The only (transitive) caller that seems to be interested in the format
is the API endpoint for content listing.

The warning about not being able to query in the expected format might
not be seen by consumers that only use the API result, so this helps
admins detect such images. It is also for future-proofing, should any
new callers want to use only images of certain formats to error out
early.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2024-12-09 08:55:10 +01:00
Fiona Ebner
936fbe3f35 file size info: prepare to make format parameter mandatory
In almost all cases, there already is an expected format. Allow
setting a special value 'auto-detect' to opt-in to automatic format
detection by 'qemu-img' for the exceptions.

Suggested-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2024-12-09 08:55:10 +01:00
Fiona Ebner
9612844ff3 btrfs: list images: specify format when querying file info
This avoids auto-detection by qemu-img and so the information will be
correct with respect to the actual image format on the storage layer.

Should the image not be in the correct format, warn and try again
querying as raw, so the image is still listed. The image is present,
so it is better if it is listed and for some backwards compatibility.
The format is still returned as the matched format in such a case,
because that is how the image is treated, even if corrupt.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2024-12-09 08:55:10 +01:00