Commit Graph

50 Commits

Author SHA1 Message Date
Thomas Lamprecht
81a90b7991 file/volume size info: add actual errors to untaint messages
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
(cherry picked from commit d4e00f2bd5)
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-23 08:41:06 +02:00
Stoiko Ivanov
545288638b plugins: untaint volume_size_info retuns
the size returned by volume_size_info is used for creating the new
destination image in PVE::QemuServer::clone_disk (and probably
elsewhere). In certain cases the return values are tainted - they are
obtained by a run_command call and depending on the format and length
of the parsed output can still have their tainted attribute.

One example of a tainted return has been reported in our
community-forum:
https://forum.proxmox.com/threads/cannot-clone-vm-or-move-disk-with-more-than-13-snapshots.89628/

A qcow2 image with 13 snapshots generates a output > 4k in length from
`qemu-img info --output=json`, which in turn causes the output to be
considered tainted.

This patch untaints the returns where applicable. The other
storage-plugins are not affected:
* LVMPlugin returns a single number and a newline (thus gets untainted
  by run_command)
* RBDPlugin untaints the complete json before decoding
* ZFSPoolplugin and ISCSIDirectPlugin explicitly untaint their
  returns.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
(cherry picked from commit ac598d851e)
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-23 08:41:06 +02:00
Thomas Lamprecht
345243fa7e tree-wide: fix typos with codespell
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
(cherry picked from commit ffc31266da)
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-23 08:41:05 +02:00
Fabian Grünbichler
71a531b63c pbs: fix typo
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
(cherry picked from commit 5b955999b9)
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-23 08:39:38 +02:00
Fabian Grünbichler
c56f7a71af pbs: allow setting up a master key
similar to the existing encryption key handling, but without
auto-generation since we only have the public part here.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-04-22 21:56:31 +02:00
Fabian Grünbichler
dfa374d320 fix #3363: avoid undef-warning for PBS crypt-mode
it is optional after all, and missing (/None) for files stored in the
snapshot dir but not referenced in the manifest for whatever reason.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-03-31 10:22:52 +02:00
Dominik Csapak
53003cb5ea PBSPlugin: use get_repository from PVE::PBSClient
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-12-03 16:53:53 +01:00
Wolfgang Bumiller
ab90c3b1f1 pbs: fix token auth with PVE::APIClient
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-12-03 16:53:43 +01:00
Thomas Lamprecht
2cd10f58be pbs: activate storage: fully validate if storage config works
improves UX of on_update and on_add hooks *a lot*.

This is a bit more expensive than the TCP ping, or even just an
unauthenticated ping, but not as bad as a full datastore status - as
this only reads the datastore config file (which is normally in page
cache anyway).

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-01 19:22:47 +01:00
Thomas Lamprecht
8b62ac6a0c pbs: add scan datastore helper
for use in both, the scan API and the on_add/on_update hooks

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-01 19:22:47 +01:00
Thomas Lamprecht
2f9eb6dc4c pbs: reuse pve apiclient for api connect helper
it is flexible enough to easily do so, and should do well until we
actually have cheap native bindings (e.g., through wolfgangs rust
permlod magic).

Make it a private helper, we do *not* want to expose it directly for
now.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-01 19:22:47 +01:00
Fabian Ebner
f3ccd0ef3d plugin: hooks: add explicit returns
to avoid returning something unexpected. Finish what
afeda18256 already started for all the other
plugins. At least for ZFS's on_add_hook this is necessary (adding a ZFS storage
currently fails as reported here [0]), but it cannot hurt
in the other places either as the only hooks we expect to return something
currently are PBS's on_add_hook and on_update_hook.

[0]: https://forum.proxmox.com/threads/gui-add-zfs-storage-verification-failed-400-config-type-check-object-failed.79734/

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-11-27 10:45:42 +01:00
Thomas Lamprecht
878fe0177e api: content: pass encrypted status for PBS backups
Prefer the fingerprint, fallback to checking the files crypt-mode.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-24 23:18:05 +01:00
Thomas Lamprecht
3cc2eb738f pbs add/update: save fingerprint in storage config
fallback to the old truthy "1" if not available

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-24 22:09:40 +01:00
Thomas Lamprecht
d2c47b3837 pbs add/update: do basic key value validation
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-24 22:09:15 +01:00
Thomas Lamprecht
478609d3bc pbs: autogen key: rename old one if existing
it could be debated do have some security implications and that
deletion is safer, but key deletion is a pretty hairy thing.

Should be documented, and people just should use delete instead of
autogen if they want to "destroy" a key.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-24 22:05:23 +01:00
Dominik Csapak
45e93e6dda Storage/PBSPlugin: implement get/update_volume_notes for pbs
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-24 13:42:13 +01:00
Dominik Csapak
6fef456c8d rename comment to notes
so that we are more consistent with pbs

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-24 10:23:25 +01:00
Fabian Ebner
1b87f01388 prune: introduce keep-all option
useful to have an alternative to the old maxfiles = 0. There has to
be a way for vzdump to distinguish between:
1. use the /etc/vzdump.conf default (when no options are configured for the storage)
2. use no limit (when keep-all=1)

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-11-23 15:27:17 +01:00
Fabian Ebner
14c922b7da don't pass along keep-options equal to zero to PBS
In PBS, zero is not allowed for these options.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-11-16 10:14:11 +01:00
Thomas Lamprecht
0b6b98d189 pbs: add/update: return enc. key, if newly set or auto-generated
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-12 18:05:28 +01:00
Dominik Csapak
9778e5c216 api: content listing: add comment and verification fields
for now only for PBS, since we do not have such info elsewhere

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-12 17:16:55 +01:00
Thomas Lamprecht
4558cb6eb6 pbs: autogen encryption key: bubble up error message
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-12 11:49:01 +01:00
Thomas Lamprecht
3de423680a PBS: use simple TCP ping for online check for now
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-02 15:55:03 +02:00
Thomas Lamprecht
4133e6e216 PBS: add support to specify port
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-02 15:49:48 +02:00
Thomas Lamprecht
4ef17e1fe0 pbs: check if encryption key could be deleted
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-24 19:06:09 +02:00
Thomas Lamprecht
18cf6c9f27 config: add hint for set encryption key, die on unlink error
as else the API cannot easily know if this is set, it cannot check
with -f as the key is in a restricted area and we do not want that a
GET runs as protected.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-24 19:05:25 +02:00
Fabian Ebner
8f26b3910d Add prune_backups to storage API
Implement it for generic storages supporting backups
(i.e. directory-based storages) and add a wrapper for PBS.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-07-24 15:44:53 +02:00
Fabian Ebner
3353698f45 Introduce prune-backups property for directory-based storages
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-07-24 15:44:53 +02:00
Wolfgang Bumiller
02cc5e1033 quick fixup for prune command in vzdump
This is a hack and we should get rid of `run_client_cmd` and
`run_raw_client_cmd` as an API entry!

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-09 14:55:07 +02:00
Thomas Lamprecht
bb0a0f9680 pbs: activate_storage: query status to ensure we can connect
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-09 11:54:13 +02:00
Thomas Lamprecht
1aeb322bc3 pbs: allow to autogen an encryption key
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-09 11:53:51 +02:00
Thomas Lamprecht
ce2e273359 pbs: rename encryption_key param to encryption-key
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-09 11:53:27 +02:00
Wolfgang Bumiller
72385de9e2 refactor sensitive parameter handling
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-09 11:49:02 +02:00
Wolfgang Bumiller
76bb5febc7 pbs: encryption support, split "raw client command" API
(And deprecate it...)

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-09 11:49:02 +02:00
Thomas Lamprecht
e6d1edcb93 pbs: run_raw_client_cmd: fix typo, and drop repetive info
It would be s/bps/pbs/ but as we already have "proxmox-backup-client"
included in the log through the executable name, so it should be
clear that this is a PBS command - so drop that part entirely.

Now using:
> run: /usr/bin/proxmox-backup-client ...

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-07 10:15:59 +02:00
Fabian Ebner
8602fd5646 PBSPlugin: Add print_volid helper
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-06-30 13:59:23 +02:00
Fabian Ebner
ddf7fdaa82 PBSPlugin: list_volumes: filter by vmid if specified
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-06-06 19:38:02 +02:00
Dietmar Maurer
8b4c2a7ec6 PBSPlugin.pm: avoid ascii-art in proxmox-backup-client output 2020-06-03 11:07:01 +02:00
Thomas Lamprecht
c05b1a8cb9 PBS plugin: code cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-06 19:26:45 +01:00
Dietmar Maurer
545e127e52 PBS Plugin: list_volumes: add ctime
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2020-03-06 19:26:45 +01:00
Dietmar Maurer
b494636ac9 PBSPlugin.pm: fix password handling using new on_update_hook 2020-02-20 12:42:59 +01:00
Thomas Lamprecht
9e34813f6c pbs: ensure storage secret file directory exists
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-02-20 11:12:39 +01:00
Dietmar Maurer
bb8adeb226 PBSPlugin.pm - extract_vzdump_config: fix call to run_raw_client_cmd 2020-02-20 10:45:34 +01:00
Thomas Lamprecht
462537a270 namespace storage specific secret files to 'priv/storage' folder
As /etc/pve/priv is already pretty polluted, having a
"<storage-id>.pw" file there smells like it could make problems in
the future.

So let the pbs pw file generator use /etc/pve/priv/storages as base
path.
Other storage should move also to that path in the future, if they
save such secrets anywhere in /etc/pve.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-02-19 15:00:54 +01:00
Thomas Lamprecht
1574a590a5 check if client executable ist installed before running command
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-02-19 14:50:42 +01:00
Thomas Lamprecht
fee2ece310 use one liner closure for outfunc
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-02-19 14:50:04 +01:00
Thomas Lamprecht
f155c912d0 indentation fixes
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-02-19 14:49:38 +01:00
Dietmar Maurer
c855ac150c implement extract_vzdump_config for PBSPlugin 2020-02-19 14:00:04 +01:00
Dietmar Maurer
271fe39460 PVE/Storage/PBSPlugin.pm: start new proxmox backup server plugin 2020-02-19 14:00:04 +01:00