Commit Graph

127 Commits

Author SHA1 Message Date
Thomas Lamprecht
ad436fabdf buildsys: avoid that build directory is created by phony target
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-10-17 08:33:28 +02:00
Fabian Grünbichler
a0a1cae4b4 cli: allow listing snapshots of all mirrors
and slightly change the output format accordingly.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-26 09:29:51 +02:00
Fabian Grünbichler
d056f823f0 medium: add diff command
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-26 09:29:49 +02:00
Fabian Grünbichler
529111dc6c snapshots: add diff command
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-26 09:29:47 +02:00
Fabian Grünbichler
5cf7a98f42 pool: add diff and list helpers
one for diffing two relative paths within a pool (e.g., for comparing
snapshots), one for diffing two pools (e.g., for diffing mirror and
mirror on medium), and one for listing paths.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-26 09:29:43 +02:00
Wolfgang Bumiller
5ce9ab44e8 clippy fixups
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-09-23 14:29:23 +02:00
Wolfgang Bumiller
36dfc65037 error handling cleanup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-09-23 14:24:49 +02:00
Fabian Grünbichler
9213b79a61 mirror: collect and summarize warnings
the output can get quite long and warnings can easily be missed
otherwise.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-23 14:18:04 +02:00
Fabian Grünbichler
96a804150b fix #4259: mirror: add ignore-errors option
to make fetching errors from broken repositories non-fatal.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-23 14:18:03 +02:00
Thomas Lamprecht
d0e605af5f bump version to 0.4.0-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-09-16 14:26:22 +02:00
Fabian Grünbichler
ffc3ed4e06 cli: add mirror snapshot create-all command
that creates a new snapshot for each configured mirror, collecting the
results and printing a summary at the end. this should be suitable for
usage in a cron job or timer-triggered unit, with no output on stderr
for 100% OK execution runs.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-16 14:21:57 +02:00
Fabian Grünbichler
047248215e cli: extract subscription key helper
for re-use in the (not-yet-created) create-all command.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-16 14:21:57 +02:00
Fabian Grünbichler
d2757931b2 mirror: add --dry-run parameter
in dry-run mode, creating a snapshot will download (but not persist) the
Release files and any indices referenced within, but not download the
package files themselves. instead, any URLs that would still need to be
fetched are printed, and the statistics about to-be-fetched files and
bytes is updated accordingly.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-16 14:21:57 +02:00
Thomas Lamprecht
91bd534701 cargo: bump proxmox-apt dependency to 0.9.2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-09-16 14:20:37 +02:00
Fabian Grünbichler
c5fed38dd8 mirror: handle indices which are only available compressed
there are repositories out there that not only skip serving the
uncompressed version, but not even reference it in their Release
file(s).

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-16 14:14:14 +02:00
Fabian Grünbichler
8a876c0117 mirror: use new architecture helper
in order to avoid having a list of arch-specific references on two
places.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-16 14:14:14 +02:00
Fabian Grünbichler
8063fd368d mirror: support acquiring indices by hash
requires proxmox-apt > 0.9.1, since earlier versions misdetect by-hash
support in the release file.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-16 14:14:14 +02:00
Fabian Grünbichler
7829ab74ed mirror: skip failed, non Packages references
these contain extra data that is not that important for the main
repository use case - providing deb packages.

if they are not retrievable (e.g., Ubuntu *only* provides some of they
via by-hash, which proxmox-offline-mirror doesn't yet support) a warning
should be enough, instead of failing the whole snapshot creation.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-16 14:13:48 +02:00
Fabian Grünbichler
bb1685a070 mirror: use xz multi decoder
Ubuntu's Packages.xz files require it, because they contain multiple
streams.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-16 14:13:48 +02:00
Thomas Lamprecht
5b230fb998 docs: expand/structure using medium section
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-09-15 14:11:54 +02:00
Thomas Lamprecht
c0d6f18b0a buildsys: clean: remove all $(PACKAGE)-*/ directories
to ensure we also get the old, left over ones on a new version bump

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-09-15 11:39:02 +02:00
Thomas Lamprecht
eb724f4021 buildsys: add lintian-overrides for shipped javascript libraries
albei we may want to really reuse the system ones..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-09-15 11:37:49 +02:00
Thomas Lamprecht
6cbd79fcb7 d/control: fix line length of helper description
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-09-15 11:29:25 +02:00
Thomas Lamprecht
95c070a954 buildsys: pass all relevant debs to linitan
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-09-15 11:18:47 +02:00
Thomas Lamprecht
cde5983fa3 docs: offline keys: add info about minimum product version that got activation support
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-09-15 11:05:35 +02:00
Thomas Lamprecht
3be91c353e d/control: fix font build-dependencies for docs
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-09-15 10:31:32 +02:00
Thomas Lamprecht
46d31930c0 docs: offline media: fix source text width to 100cc
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-09-12 10:33:48 +02:00
Stoiko Ivanov
f5d631e307 docs: add minimal sample nginx config
if users want to use their offline medium via http in their internal
network.

nginx config is based off the one we use for download.proxmox.com.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2022-09-12 09:39:59 +02:00
Fabian Grünbichler
141cdb6986 build: track docs build-dependencies
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-09 14:48:18 +02:00
Fabian Grünbichler
d83e5ea719 docs: fix too-short title underline
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-09 14:14:17 +02:00
Fabian Grünbichler
8791f19e00 bump version to 0.3.0-1
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-09 14:10:16 +02:00
Fabian Grünbichler
308dbe109d docs: correct CLI titles
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-09 14:10:16 +02:00
Fabian Grünbichler
466d72aa6d rename proxmox-apt-repo to proxmox-offline-mirror-helper
and move it into its own package

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-09 14:04:29 +02:00
Fabian Grünbichler
5b09cb27a7 pool sync: improve progress report
print at least once every 30s to avoid no progress at all on small
mirrors that are part of a big pool.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-09 14:04:29 +02:00
Fabian Grünbichler
01ab3f0474 docs: add snapshot term
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-09 14:02:05 +02:00
Fabian Grünbichler
6f59b35dca docs: adapt to pool/mirror changes
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-09 14:02:05 +02:00
Thomas Lamprecht
240307a92b docs: intro: avoid office@ mail address for sales related questions
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-09-09 13:32:04 +02:00
Aaron Lauterer
e5fe7f7cea docs: offline-media: improve read- and understandability
by rephrasing parts to be easier to read or adding more context.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2022-09-09 12:38:10 +02:00
Aaron Lauterer
07fae362b6 docs: offline-keys: improve read- and understandabilty
By rephasing certain parts and splitting long sentences into smaller
ones.

Also changing "licensing server" to "subscription server".

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2022-09-09 12:38:10 +02:00
Aaron Lauterer
a1bae05459 docs: installation: improve read- and understandabilty
This patch contains the larger changes, such as splitting longer
sentences and rewording some parts.

The headline for "Reuse a Proxmox Repository for Offline Mirror Tool"
has been changed to "Set up the Repository on non Proxmox bases
systems". Sounds a bit nicer and we are in the Proxmox Offline Mirror
docs anyway.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2022-09-09 12:38:10 +02:00
Aaron Lauterer
32f3eca628 docs: installation: fix typos, and other small changes
switch from `Proxmox projects` to `Proxmox solutions`.
Add missing words, such as `the` and drop `also`s.

And other small changes that don't affect the actual content or meaning.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2022-09-09 12:38:10 +02:00
Aaron Lauterer
49c845cde5 docs: introduction: rewrite for better readability
splitting long sentences, rephrasing and extending some parts

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2022-09-09 12:38:10 +02:00
Aaron Lauterer
41cb625b6e docs: introduction: fix typos and missing words
adding `the` in a few places, dropping `also`s as they are almost always
just filler words.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2022-09-09 12:38:10 +02:00
Thomas Lamprecht
ac8d260964 add .gitignore
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-09-09 12:38:08 +02:00
Fabian Grünbichler
0dcf473c40 pool: remove double quoting
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-09 12:00:35 +02:00
Fabian Grünbichler
c598cb154e rework pool handling
instead of having a .pool directory inside each mirror directory (both
on the mirror and medium side), instead

- specify a base directory for each mirror which can be shared
- create a .pool in this base directory
- create the mirror dir named after the mirror ID in this base directory
- adapt GC, wizard, .. accordingly

to re-use a shared pool on the medium side, hash the source mirror base
dir path to derive the destination pool .pool-XX directory name and
persist the resulting relative path in the medium state file.
2022-09-09 12:00:35 +02:00
Fabian Grünbichler
ff3d11a5d8 pool: allow pool re-use
and improve the naming of the two dirs.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-09 11:28:52 +02:00
Fabian Grünbichler
54c8397731 clippy fix
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-09 10:06:46 +02:00
Stoiko Ivanov
5d9224ed34 fix typos in command invocation
noted while blindly pasting the output after setting up my first
mirror.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2022-09-08 15:50:52 +02:00
Thomas Lamprecht
5a813b796a buildsys: dinstall target: install doc instead of lib package
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-09-08 15:50:40 +02:00