In some situations, such as backing up VMs with 2 or more disks to
PBS, we get passed the backup archive size as a string instead of
as an integer. This led to errors rendering the notification template
down the line.
This commit explicitly casts the data from the task table to an int.
It would be a good idea to actually hunt down the places that produced
the string instead of an integer, but as a quick fix and as a
safeguard against similar lurking errors this approach is fine, IMO.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Virtual (or anonymous) endpoints/groups are used for sending
one-off notifications to a target that does not exist in the
config.
VZDump uses this to send out notification mails to those addresses
configured by the `mailto` parameter.
Suggested-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
... instead of using sendmail directly.
If the new 'notification-target' parameter is set,
we send the notification to this endpoint or group.
If 'mailto' is set, we add a temporary endpoint and a
temporary group containg both targets.
This commit also refactors the old 'sendmail' sub heavily:
- Use template-based notification text instead of endless
string concatenations
- Removing the old plaintext/HTML table rendering in favor of
the new template/property-based approach offered by the
`proxmox-notify` crate.
- Rename `sendmail` sub to `send_notification`
- Breaking out some of the code into helper subs, hopefully
reducing the spaghetti factor a bit
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
This reverts 7420d7ff ("zstd: add --rsyncable flag")
That flag causes severe slow downs on fast disks, and we still have
other rsyncable compressors available.
It was originally added based on wrong documentation that made the
performance impact look a lot smaller than it actually is.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
we want to move the 'exclude-path' to an array format (from 'string-alist')
prepare the code that it can be either a string or a list
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Since all tasks succeeded, previously no mail was sent in that case.
Note that the error passed to $self->sendmail() is added to the
subject of the mail if it is a single line or the beginning of the
mail otherwise. Thus changing the mail slightly compared to previously
for the case where the job-start hook fails and the case where the
job-end hook fails with mailnotification=always. But can be considered
an improvement, because the user sees the error right away.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Also generalizes the way vzdump property strings are handled for jobs.
Something similar could be done in VZDump.pm, but there the maxfiles
and prune-backups settings are currently coupled, so a dedicated
parse_performance() is used instead. Can be changed once maxfiles is
dropped.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
to avoid warnings like
parse error in '/etc/vzdump.conf' - 'storage': missing property -
'notes-template' requires this property
when there is no default for the required property configured.
In new(), the defaults are mixed in with the regular CLI/API
parameters, so re-check if the required property is set. If it's not,
the defaults do not apply to the current run, and can be dropped.
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
For VMs, $task->{hostname} might be undef and when running on a
stand-alone node, there is no cluster name.
Reported-by: Marco Gabriel <mgabriel@inett.de>
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
instead of just checking for a newline, do a full check already.
Also do the check at the beginning of generate_notes() for consistency
and remove the check after expansion to avoid failing late for things
like '{{cl{{node}}er}}' (which can even expand to a valid variable
making the error even more confusing).
Co-developed-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
so that '{{foo}}{{bar}}' is not detected as being an unknown variable
named 'foo}}{{bar', but as 'foo' (and 'bar').
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
which is caused by the quoting operators \Q...\E. The actual intention
was to avoid such surprises.
Fixes: e01438a7 ("partially close#438: vzdump: support setting notes-template")
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Previously, if the '--script' argument was passed with a non-existent
file, it would state that a non-executable script was the reason for
failure. This adds a check to see if the hook script exists, in order
to provide a more accurate error message.
Also adds an 'Error:' prefix the 'script not executable' error.
Signed-off-by: Dylan Whyte <d.whyte@proxmox.com>
While vzdump itself wouldn't mind about unescaped newlines, the
parameter isn't supposed to contain any, and when used as part of the
job config, it has to be a single line too, so make it consistent.
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Check the number of protected backups early if the protected flag
is set.
Suggested-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
else this single call site is subtly different from all the rest, which
could cause problems further down the line if we ever change the prune
logic.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
since they are the ones relevant for pruning and protected backups
have their own separate limit.
Since get_backup_file_list is only used in places where the
unprotected backups are needed, adapt the helper accordingly.
If there is a storage, use PVE::Storage::volume_list to count the
unprotected backups. This avoids a direct invocation of the
proxmox-backup-client for PBS and the limit check can also work for
external storage plugins which might not be dir-based or name the
backups differently.
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
which allows users to prepare the backup storage for activation (e.g.
by waking up a remote node).
Suggested-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Reviewed-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Acked-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Keeps the behavior consistent with what happens for storages. It also
is required to not get into conflict with the check in archive_remove,
i.e. pruning here marks a backup as 'remove' and then archive_remove
complains that it's protected.
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
which ended up in the backup task log and can be confusing to users:
> INFO: Backup finished at 2021-10-11 08:40:38
> Result: {
> "data": null
> }
> INFO: Backup job finished successfully
For proxmox-backup-server < 2.0.11-1, there was no output for the
upload-log command.
Reported in the community forum:
https://forum.proxmox.com/threads/backup-finishes-ok-but-data-null-info-on-finish.97740/
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Not doing so leads to an off-by-one error, and pruning in a dumpdir
really should behave the same way as pruning in a storage.
Reported in the community forum:
https://forum.proxmox.com/threads/problem-mit-prune-backups.95737/
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
remove type check for the specific plugins, instead we can deduce it
from the supported content type in config (this can only be set on
storages that declare to support backups). should also work with
external storage plugins.
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
may not be obvious from the task log alone if we're relying on the
retention setting configured on a storage or the ones from local
node's vzdump.conf
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
if passing the hook script command as string, it might get interpreted
as shell command with side-effects. this is pretty harmless, since only
root is allowed to set the script parameter anyway, but making it more
robust and future-proof does not hurt.
tested with a reproducer of "/bin/echo $(touch $(whoami))" as script
parameter, with a file with that name existing, being executable and
having the following contents:
----8<----
echo "hello from hook script"
---->8----
without this change, the hookscript itself is not executed, but
'/bin/sh -c "/bin/echo $(touch $(whoami)) job"' and similar calls are,
which cause the file 'root' to be touched in the current working
directory of the vzdump process (or task worker).
with this change, the file is executed as is without any side-effects of
shell commands in the file name, and the 'hello from hook script' lines
are printed whenever the hook script is called by vzdump.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
so it doesn't get out of scope too early.
Regression introduced by 5620e5761e as pointed
out by Fabian Grünbichler.
Reported in the community forum:
https://forum.proxmox.com/threads/limit-simultaneous-backup-jobs.87489
Suggested-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Otherwise storage_info() cannot be used for (at least) PBS storages from an API
call without 'protected => 1', because the password cannot be read from
'/etc/pve/priv'. Note that the function itself does not need the storage to be
active, because it only uses storage_config() and get_backup_dir().
AFAICT new() is the only existing user of this function and can be responsible
for activating the storage itself.
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
The initial default from the $confdesc is 1 anyways, and like
this changing the default in /etc/vzdump.conf to 0 actually works.
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
reported by perlcritic. The bareword file handles will be depreacated
in the future in perl, so start replacing now..
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
When a job is updated, verify_vzdump_parameters() is called twice. This led to
parse_property_string being called with the already parsed option.
Reported on the pve-user mailing list:
https://lists.proxmox.com/pipermail/pve-user/2021-January/172258.html
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Errors from storage_info() are newline-terminated, because perl would append
the line number otherwise. Chomp those errors, because sendmail() relies
on the presence of a newline to decide if it's multiple problems or only one.
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
