Commit Graph

222 Commits

Author SHA1 Message Date
Thomas Lamprecht
2971ba9487 d/control: add missing ${misc:Depends}
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-30 13:51:35 +02:00
Thomas Lamprecht
44c5a93709 fixup! makefile: convert to use simple parenthesis 2023-05-30 13:51:07 +02:00
Thomas Lamprecht
426271c059 avoid debcargo on build
Switch over to a more manual build oriented on proxmox-backup and
pve-lxc-syscalld.

nothing against debcargo directly, but currently it's really tailored
to Debians re-package upstream crate needs, and that shows.

Once we can generate control files from it without requiring all
build dependencies, and potentially some other stuff fixed, we could
re-introduce it.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-30 13:50:40 +02:00
Thomas Lamprecht
758d000412 drop librust-proxmox-offline-mirror-dev package
was only auto-generated by debcargo/dh-cargo but we never uploaded it
nor used it anywhere FWICT

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-30 13:45:12 +02:00
Thomas Lamprecht
8c7f5fb92a d/control: define compat level via build-depends and raise to 13
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-30 13:10:47 +02:00
Thomas Lamprecht
ea9b44c98f update dependencies for current bookworm state
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-30 13:10:36 +02:00
Thomas Lamprecht
d5c4f90d88 makefile: convert to use simple parenthesis
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-30 12:24:45 +02:00
Thomas Lamprecht
65e33b4673 debian: wrap-and-sort -tkn
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-30 11:24:36 +02:00
Thomas Lamprecht
23712e9e08 cargo fmt
seems I forgot to save the vim buffer after running this inline..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-04-24 11:47:01 +02:00
Fabian Grünbichler
bc4acb26a2 fix #4614: add note about key requirements to mirror docs
and reference the key part of the documentation.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-04-24 11:38:17 +02:00
Fabian Grünbichler
28945c9a71 setup wizard: add subscription keys
to make it a bit easier to configure access to the enterprise repositories.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 [ T: added missing use statement for format_err ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-04-24 11:38:07 +02:00
Fabian Grünbichler
6b10e0eb61 fix compat with proxmox-router
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-04-17 14:32:59 +02:00
Fabian Grünbichler
9de6cbda14 drop already finished TODO
we did settle on 'pom' as key prefix in the end.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-04-06 13:54:20 +02:00
Fabian Grünbichler
50d37e1b3c config: properly skip serializing empty weak_crypto
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-04-06 13:53:43 +02:00
Fabian Grünbichler
8282ee4790 fix #4261: allow GC for all configured mirrors
by making the --id parameter optional, and structuring the output accordingly.
since pools are per base-dir, GC only needs to run once per base-dir instead of
for each mirror entry.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-04-06 13:23:32 +02:00
Fabian Grünbichler
e361735f23 pool: drop redundant quote for paths..
a Path(Buf)'s Debug formatting already contains double quotes, no need to quote
it a second time..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-04-06 13:23:32 +02:00
Fabian Grünbichler
fa95f21ab4 fix #4632: allow escape hatches for legacy repositories
there are still repositories out there that are using things like DSA/RSA-1024
and SHA1, so let's allow POM users to opt into accepting those insecure
cryptographic parameters, but keep the default settings secure.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-04-06 13:19:56 +02:00
Fabian Grünbichler
c71d434935 improve GPG error messages
e.g., when encoutering a key that is self-signed with SHA-1 (which is not that
uncommon for non-distro repositories that have an old key), instead of the
following:

----8<----
Fetching Release/Release.gpg files
-> GET 'https://download.ceph.com/debian-quincy//dists/bullseye/Release.gpg'..
-> GET 'https://download.ceph.com/debian-quincy//dists/bullseye/Release'..
Verifying 'Release(.gpg)' signature using provided repository key..
	Subkey of 08B73419AC32B4E966C1A330E84AC2C0460F3994 not bound: No binding signature at time 2022-10-17T22:41:10Z
Error: encountered 1 error(s)
---->8----

which only gives us a rought idea that something is wrong with a key signature,
we now get the following:

----8<----
Fetching Release/Release.gpg files
-> GET 'https://download.ceph.com/debian-quincy//dists/bullseye/Release.gpg'..
-> GET 'https://download.ceph.com/debian-quincy//dists/bullseye/Release'..
Verifying 'Release(.gpg)' signature using provided repository key..

Subkey of 08B73419AC32B4E966C1A330E84AC2C0460F3994 not bound: No binding signature at time 2022-10-17T22:41:10Z
Caused by:
	0: Policy rejected non-revocation signature (PositiveCertification) requiring second pre-image resistance
	1: SHA1 is not considered secure since 2023-02-01T00:00:00Z

Error: No valid signature found.
---->8----

which shows us that the key signature was rejected because it's SHA-1, and the
(default and currently only) policy doesn't allow that (anymore).

the output is also improved in case the Release file is signed multiple times
and none of the signatures are accepted.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-04-06 13:19:56 +02:00
Fabian Grünbichler
ab7b070305 actually include version in user-agent
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-03-31 11:14:11 +02:00
Fabian Grünbichler
29c23dc097 update d/control
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-03-03 12:16:26 +01:00
Fabian Grünbichler
effa25be25 update proxmox-http to 0.8
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-03-03 10:51:45 +01:00
Thomas Lamprecht
ebb5569215 d/control: update
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-02-07 15:41:59 +01:00
Thomas Lamprecht
8497b4fe22 bump version to 0.5.1-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-02-07 15:36:03 +01:00
Lukas Wagner
9212c1fde5 docs: document ALL_PROXY environment variable
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2023-01-27 10:02:23 +01:00
Lukas Wagner
7fdd1a3dee fix #4445: mirror: subscription: add proxy support
This commit adds support for HTTP proxies, configurable via the
ALL_PROXY environment variable.

For example:
  $ ALL_PROXY="localhost:3128" proxmox-offline-mirror mirror <...>

Note: `ureq` seems to use HTTP CONNECT for *all* connections, including
HTTP on port 80. Proxies need to be configured to allow that - Squid by
default allows CONNECT only for HTTPS on port 443.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2023-01-27 10:02:12 +01:00
Fabian Grünbichler
26e97f4513 add missing build dependencies
found by building with sbuild in a clean chroot.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-01-05 14:19:10 +01:00
Fabian Grünbichler
404eedad0b update to nix 0.26.1
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-01-05 14:18:59 +01:00
Fabian Grünbichler
67e1283c07 update d/control
after debcargo update

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-01-05 10:19:28 +01:00
Thomas Lamprecht
ab2be48e19 bump version to 0.5.0-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-10-22 17:02:09 +02:00
Thomas Lamprecht
8b7c7967e8 guided setup: add Quincy as supported release for bullseye
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-10-22 16:37:01 +02:00
Fabian Grünbichler
a4a06e8a4f mirror wizard: remove stray ':' from filter query
the query helper already adds the final ':'.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-10-21 13:11:37 +02:00
Wolfgang Bumiller
f4d89ed79a clippy fixup
It's internal to the fn and the types are all written out
and visible so it's arguably fine this way.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-10-21 13:00:36 +02:00
Fabian Grünbichler
36e33c1aa4 helper: add status command
similar to `proxmox-offline-mirror medium status <ID>`, but limited to
the information that is stored on the medium itself. this command can be
used to get a quick overview over what's on a medium, or for automated
setup of the contained repositories.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-10-21 12:52:18 +02:00
Fabian Grünbichler
7a94bee317 helper: make mountpoint non-optional
the default/fallback was a remnant from a no-longer-employed
"copy-helper-to-medium" deployment strategy.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-10-21 12:52:13 +02:00
Wolfgang Bumiller
56ca838b56 cargo fmt
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-10-21 12:49:43 +02:00
Fabian Grünbichler
cd1d70bdf2 docs: add section/package filters
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-10-21 12:36:41 +02:00
Fabian Grünbichler
f907fd5ed1 mirror setup: query filters in guided mode
with a somewhat sensible default of filtering the games and debug
sections - which already reduces a mirror of PVE + Debian bullseye by
about 27% (105GB->77GB).

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-10-21 12:36:40 +02:00
Fabian Grünbichler
695850273a mirror: also filter {component}/{section}
so that a single filter 'games' covers all related sections for a stock
Debian repository.

this also has the side-effect that package downloads are now batched by
component.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-10-21 12:36:37 +02:00
Thomas Lamprecht
36949d116b mirror, pool: fix comment text width
rustfmt ignores those...

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-10-20 14:33:39 +02:00
Fabian Grünbichler
837867edb2 mirror: refactor fetch_binary/source_packages
and pull out some of the progress variables into a struct.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-10-19 16:25:25 +02:00
Fabian Grünbichler
7c17509f2c fix #4264: only require either Release or InRelease
strictly speaking InRelease is required, and Release optional, but that
might not be true for older repositories. treat failure to fetch either
as non-fatal, provided the other is available.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-10-19 16:25:25 +02:00
Fabian Grünbichler
67c0b91cee mirror: implement source packages mirroring
similar to the binary package one, but with one additional layer since
each source package consists of 2-3 files, not a single .deb file.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-10-19 16:25:25 +02:00
Fabian Grünbichler
e79308e626 mirror: add exclusion of packages/sections
to keep the size of mirror snapshots down by excluding unnecessary files
(e.g., games data, browsers, debug packages, ..).

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-10-19 16:25:25 +02:00
Thomas Lamprecht
05800963b1 cargo: update proxmox-apt to 0.9.3
to get new source index support and 'section' struct field in
preparation of new features

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-10-19 16:25:25 +02:00
Fabian Grünbichler
03ea3497c7 d/debcargo.toml: add missing B-D
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-10-18 10:26:29 +02:00
Thomas Lamprecht
ad436fabdf buildsys: avoid that build directory is created by phony target
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-10-17 08:33:28 +02:00
Fabian Grünbichler
a0a1cae4b4 cli: allow listing snapshots of all mirrors
and slightly change the output format accordingly.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-26 09:29:51 +02:00
Fabian Grünbichler
d056f823f0 medium: add diff command
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-26 09:29:49 +02:00
Fabian Grünbichler
529111dc6c snapshots: add diff command
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-26 09:29:47 +02:00
Fabian Grünbichler
5cf7a98f42 pool: add diff and list helpers
one for diffing two relative paths within a pool (e.g., for comparing
snapshots), one for diffing two pools (e.g., for diffing mirror and
mirror on medium), and one for listing paths.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-26 09:29:43 +02:00