Commit Graph

1326 Commits

Author SHA1 Message Date
Thomas Lamprecht
ae54eabff9 tools: followup: fix comment length and rename to upid_normalize_status_type
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-28 14:51:54 +02:00
Dominik Csapak
4e5360384c PVE/Tools: add 'upid_get_status_type'
as a single point where we get the type of upid status

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-06-28 14:47:53 +02:00
Dominik Csapak
f43ace29ca PVE/JSONSchema: add pve-task-status-type
to have a format that contains the possible worker task states

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-06-28 14:47:53 +02:00
Thomas Lamprecht
13c771819a inotify/network: indentation fix
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-24 15:36:04 +02:00
Thomas Lamprecht
0438d431e4 bump version to 7.0-3
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-23 12:08:00 +02:00
Stefan Reiter
ffe48b8e8a systemd: allow SendSIGKILL and TimeoutStopUSec dbus properties
Used in qemu-server for avoiding KillMode 'none'. SendSIGKILL is a
boolean, so we need to use dbus_boolean to serialize it.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-06-22 07:45:21 +02:00
Dominik Csapak
9d065c72fb SysFSTools: change 'product' to 'device'
so it is more consistend with the source (it comes from the file
'device') as well as the subsytem_device field

the only place we use that field is in the same file in pci_dev_bind_to_vfio,
which we also change here, so that should not be a breaking change
(in qemu-server we only really use the existance and the has_fl_reset
flag)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-06-21 17:24:06 +02:00
Dominik Csapak
aa59b1121c SysFSTools: add verbose flag to pci_device_info
to also get the subsystem_vendor and device, as well as the
iommu group and mediated device support

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-06-21 17:24:06 +02:00
Thomas Lamprecht
a03e5b39e1 bump version to 7.0-2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-17 16:41:57 +02:00
Fabian Ebner
4cc5b13dfc tools: add upid_status_is_error function
There's also support for ending a task with warnings now, so the logic "status
not 'OK' means error" does not work anymore.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-06-17 15:22:56 +02:00
Thomas Lamprecht
20000e9ff6 d/control: record new dependency for libnetaddr-ip-perl
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-17 15:20:14 +02:00
Fabian Ebner
8f75194cd8 network: add unique_ips function
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-06-17 15:17:37 +02:00
Fabian Ebner
8286ef53bb network: add canonical_ip function
Net::IP doesn't seem to have a function for it and normalizing to the full
quad-form is less then ideal if we inted to output IPv6 addresses returned by
that function at some point.

Instead, use NetAddr::IP, which is already used in pve-network.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-06-17 15:17:37 +02:00
Fabian Ebner
b0e3bcc186 network: is_ip_in_cidr: avoid warning when versions don't match
is_ip_in_cidr('fd80:1::10', '127.0.0.1/24') would result in
    Use of uninitialized value in numeric eq (==)
as overlaps() returns undef in such a case.

Note that there are (albeit few) existing callers that don't specify $version.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-06-17 15:17:37 +02:00
Fabian Ebner
123c310474 network: is_ip_in_cidr: correctly handle the CIDR being a singleton range
i.e.  is_ip_in_cidr('127.0.0.1', '127.0.0.1/32', 4) should return 1;

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-06-17 15:17:37 +02:00
Thomas Lamprecht
06c1c13f1c tools: download from url: add option to allow overriding existing files
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-16 12:43:36 +02:00
Lorenz Stechauner
f52ecff957 tools: download_file_from_url: move check for existing file outside eval
it is not necessary to include this block in the eval which when it
fails tries to unlink $tmpdest, because in the check for the existing
file $tmpdest is not used.
2021-06-16 12:14:52 +02:00
Lorenz Stechauner
43cb80c5f2 tools: download_file_from_url: adapt error messages to start at new line
the front end expects the error message to be the first part of the
last line. putting the new line at the beginning of the die message
does not work, either.

https://lists.proxmox.com/pipermail/pve-devel/2021-June/048676.html
2021-06-16 12:14:52 +02:00
Thomas Lamprecht
9fffe4bc89 inotify: also detect VLAN id from "vlan\d+" ifaces
We support also vlanX, with X being a integer for the VLAN id, as
valid vlan iface name, so support that too here.

and make the dev name check for definedness, even if "0" is currently
not really supported (officially) it is still a valid iface name for
the kernel (which takes any byte).

The VLAN id is in the range of [2, 4094] (inclusive) so defined check
is not required there.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-15 16:45:33 +02:00
Lorenz Stechauner
eca898c0c1 tools: download_file_from_url: fix typo 2021-06-15 16:24:16 +02:00
Aaron Lauterer
b818066a85 inotify: read_interfaces: add vlan-id and vlan-raw-device on dot notation vlan interfaces
Setting the vlan-id and vlan-raw-device value for vlan devices that
follow the dot notaton (interface.vlan) aligns how dot notation vlan
devices and vlan devices that use the explicit vlan-id and
vlan-raw-device options, available with ifupdown2, are represented in
API return values.

Previously the type for both was 'vlan' but only the latter showed more
details.

Setting these values here should not have any influence on how the
interfaces file is being written as these two values are already
filtered in __write_etc_network_interfaces for dot notation devices.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2021-06-15 14:41:46 +02:00
Wolfgang Bumiller
bd9eb367a0 Syscalls/Tools: add renameat2
Mostly for the ability to atomically swap files.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-06-15 14:35:26 +02:00
Thomas Lamprecht
2531c455e8 tools, rest env: sort use statements
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-15 14:24:47 +02:00
Thomas Lamprecht
dc4bc96960 tools: get_file_hash: add use statements for Digest module
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-15 14:22:18 +02:00
Thomas Lamprecht
3a94648515 tools: download_file_from_url: handle interrupts
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-15 14:20:49 +02:00
Thomas Lamprecht
60fb1c2628 tools: download_file_from_url: improve UX and avoid cyclic dependencies
plus some refactoring

* drop worker, cannot be done here (RPCEnv is in pve-access-control)
* actually output the wrong "got" hash on mismatch
* die on existing file with mismatched
* drop double array for passing cmd
* drop `/usr/bin` prefix
* adapt rename error message
* add error handling for unlinking the temp. file

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-15 14:11:09 +02:00
Thomas Lamprecht
189f0321ca tools: cleanup usage line
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-15 13:12:58 +02:00
Lorenz Stechauner
a3327ea6fb tools: add download_file_from_url
adds a common function to download arbitrary files from urls.

code is based on
manager:PVE/API2/Nodes.pm:aplinfo

Security notice: this function does not perform any permission
checking. The callee has to make sure, that only authorized users may
use this function.

Caution: This function is able to download files from internal
networks (which would not be visible/accessible from outside), the
callee needs to ensure that unprivileged (e.g., non root@pam or the
like) can only pass OK URLs (e.g., resolving to public routable IPs)

Signed-off-by: Lorenz Stechauner <l.stechauner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-15 10:21:24 +02:00
Stoiko Ivanov
bc885c89b6 Revert "daemon: add compat code for pmgproxy 6.x"
This reverts commit a3777dce67.

With the upcoming release of pmg-api 7.0 we included the changes for
configuring a LISTEN_IP, thus this compatibility code is not needed
anymore.

Quickly tested with current pmg-api master

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2021-06-15 10:19:37 +02:00
Fabian Grünbichler
4d22a9729d tests: run with static TZ value
else tests may randomly fail depending on which local TZ is set..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-19 18:08:36 +02:00
Thomas Lamprecht
658932f7f5 bump version to 7.0-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-09 17:33:35 +02:00
Thomas Lamprecht
e0e6637a00 buildsys: change upload dist to bullseye
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-09 17:33:31 +02:00
Thomas Lamprecht
30a291eae7 d/control: bump debhelper compat to >= 12
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-09 17:32:30 +02:00
Thomas Lamprecht
bb59406e83 bump version to 6.4-3
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-07 16:24:39 +02:00
Stoiko Ivanov
a3777dce67 daemon: add compat code for pmgproxy 6.x
The changes to the listening behavior introduced with PVE 6.4 break
backwardscompatibility w.r.t. listening address and logging, which
should not be changed without explictly notifying the user.

This patch re-adds the family parameter, which is still used by
pmgproxy and based on its existence creates the socket as before.

compared to the IO::Socket::IP->new call used before
390fc10dc4, the only change is the
renaming of 'LocalAddr' to 'LocalHost' (which are synonymous in
IO::Socket::IP [0])

It can simply be reverted with the release of pmg-api 7.0 (where
we'll record the change in the release-notes and upgrade-page)

[0] https://perldoc.perl.org/IO::Socket::IP
Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2021-05-07 16:19:29 +02:00
Stoiko Ivanov
2f8be3bfda daemon: explicitly bind to wildcard address.
with the recent change in pve-manager pveproxy (and spiceproxy)
try binding to '::' per default. This fails for hosts having disabled
ipv6 via kernel commandline.

Our desired behavior of binding on '::' and only falling back to
'0.0.0.0' in case this is not supported is not directly possible with
IO::Socket::IP->new (or rather by Socket::GetAddrInfo, which at least
on my system always returns the v4 wildcard-address first).

the code now binds to:
* the provided $host if not undef
* '::' if $host is not set
* '0.0.0.0' if $host is not set and binding on '::' yields undef,
  which means that it failed to create a socket which normally means
  that IPv6 is disabled

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-07 16:19:09 +02:00
Stoiko Ivanov
9449731c60 daemon: drop Domain parameter from create_reusable_socket
The Domain parameter for IO::Socket::IP is not used/needed.
It is needed to create a IP Socket when calling IO::Socket->new,
but here we call IO::Socket::IP-new directly (see [0]).

[0] https://perldoc.perl.org/IO::Socket::IP

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2021-05-07 16:14:53 +02:00
Fabian Ebner
31d4beb47c schema: check format: parse list formats as arrays
Previously, the returned value would be only the last element or undef in case
of an empty list. There's only a handful of callers of check_format() that look
at the return value and AFAICT none of the exisitng ones is for a -list format.
But best to avoid any future surprises.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-05-03 13:05:20 +02:00
Fabian Ebner
91477acefb indentation fix
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-05-03 13:05:20 +02:00
Thomas Lamprecht
08f281bd33 bump version to 6.4-2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-26 19:34:26 +02:00
Alexandre Derumier
a85812aff5 INotify: add support for dummy interfaces type
They can be used like loopback, but not limited to only 1 interface.
It's needed for bgp with multipath/ecmp to have a unique src ip

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
2021-04-26 16:12:57 +02:00
Thomas Lamprecht
1b44e6fe0f REST handler: make API return validation opt-in
It has not shown any real value in the last decade+ it was enabled,
and it can actually add quite some performance overhead. E.g., if an
API endpoint returns a few 100k of relatively simple entries we can
easily require several seconds, even tens of seconds, to run the
return validation - making it easier to run into timeouts along the
transmit path to the client.

The CLI handler has it still enabled, normally there's no timeout
there as no HTTP transmit path is involved, and d.csapak had a slight
preference for that in an off-list discussion.

The actual implementations in PMG or PVE could enable it too if
running under debug mode.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-23 17:49:52 +02:00
Thomas Lamprecht
4a6f8872a4 code cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-23 17:49:52 +02:00
Thomas Lamprecht
53f22abfc5 bump version to 6.4-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-23 14:59:57 +02:00
Fabian Ebner
ff79ee6596 allow workers to count warnings and finish tasks in a WARNINGS state
as is already supported by the UI (and PBS).

A nice bonus is that warn() can be used by both workers and non-workers. For
workers, the output is redirected/duplicated as set up by {fork,tee}_worker(),
and non-erroring workers that issued a warning will end in a WARNINGS state.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-04-23 14:35:55 +02:00
Thomas Lamprecht
6b00e70cd1 pbs: keep a separate $USE_CRYPT_PARAMS list per command exe
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-23 14:15:13 +02:00
Thomas Lamprecht
76ddb876d9 pbs: rework client exe handling and error message
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-23 14:14:36 +02:00
Stefan Reiter
f7c0254131 PBSClient: use crypt params for file 'list' and 'extract'
Necessary for accessing encrypted backups.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-04-22 18:17:35 +02:00
Stefan Reiter
77e402f085 PBSClient: add file_restore_extract function
*_prepare creates a fifo for streaming data back to clients directly,
filefile_restore_extract blocks and should be called from a background
worker - while it is running outcoming data can be read from the FIFO.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-04-22 18:17:35 +02:00
Stefan Reiter
67252649d9 PBSClient: add file_restore_list command
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-04-22 18:17:35 +02:00