In Chrome OS /etc/machine-id is not available on the sandbox
environment. Chrome OS specifically rotate it constantly and is
considered PII.
Change-Id: Ia4113affa68c5973b36d196a35278a3f6cee2898
Signed-off-by: Daniel Campello <campello@chromium.org>
Replace fu_common_cab_build_silo() with an actual GObject that can hold parsing
state. This cleans up the code a lot, and means we can add additional
functionality in the future without breaking ABI or API.
The long term plan is to verify the metadata and payload signatures when
parsing FuCabinet, rather than much later in _check_requirements().
This of course requires passing in a keyring context (which we don't yet have)
and would mean we can stop setting the various confusing 'fwupd::ReleaseBlob'
XbNode extra data.
No logic changes for now, just a lot of moving things into sane places.
If we say that the version format should be the same for the `version_lowest`
and the `version_bootloader` then it does not always make sense to set it at
the same time.
Moving the `version_format` to a standalone first-class property also means it
can be typically be set in the custom device `_init()` function, which means we
don't need to worry about *changing* ther version format as set by the USB and
UDev superclass helpers.
This does 'bleed' the metadata contents into areas previously covered by quirks,
but in this case may be pragmatic and more up to date than a build-time
generated quirk file, which increases the user-friendliness of fwupdmgr.
In 1de7cc we checked the version format when checking for update, but there are
many other places that are doing verfmt-insensitive comparisons. For instance,
the predicates in <requires> all fail if the device version format is plain.
his breaks updating some NVMe drives where the `ne` requirements are not
semantic versions.
To avoid trying to catch all the bugs in different places, and in case we have
a future verfmt that should be treated another way, refactor this out in to a
common function and deprecate the old function.
If the device replugs, then we were doing `g_set_object(&device, device_tmp)`
which worked, but had the unfortunate effect of not unwinding properly as
`device` was a function parameter rather than a local variable.
The observed effect using `watch ((GObject*)$1)->ref_count` was the refcount was
double-decremented and thus we got a critical warning on the final valid unref.
Fix this by copying the function parameter to a local variable which we can
then change if the device replugs.
The PEM data is larger than 2kB, which was a crazy-small max size for a
signature. Make this a more respectable 1Mb so that we can load PKCS-7
signatures with full certificate info included.
This wasn't being tested in the self tests as it only affects data read from
the remote file descriptor.
This allows it to be refreshed anytime the daemon updates rather
than once a day by a systemd job.
As part of this change, remove the logging from `fwupdmgr` which
was only used for motd purposes.
Some hardware does not handle upgrading from version 1.2.2 to 1.2.4 and instead
needs to be upgraded from 1.2.2->1.2.3->1.2.4 so that on-device metadata can be
migrated correctly.
Add a new per-device flag `install-all-releases` which causes the daemon to not
skip directly to the newest release. This is designed to be set from a quirk
file.
This can obviously only be used for devices that can apply firmware "live" and
thus do not need a reboot or system shutdown to actually apply the firmware.
This also needs the cabinet archive to ship multiple versions of the firmware,
and for the metainfo.xml file to refer to multiple release objects.
These messages are useful for debugging but not to users. Many issues
get filed because they're cryptic and users can't do anything about them.
Downgrade the actual messages to debugging in the engine, and just send
back a generic message for clients.
Fixes: #1659Fixes: #1261
It's confusing to have FuConfig load both the daemon.conf file and also keep
track of the enabled remotes. It's also wasteful of memory to keep the GKeyFile
alive the entire time.
Logically these are different pools of information and should be managed by
different objects. This allows us to implement reload() in a sane way and be
less reliant on the inotify event.
Add various fixes to enable us to build a selection of useful USB plugins.
Also, skip tests that don't make sense on WIN32 or that will not work.
With much help from Mario Limonciello <mario.limonciello@dell.com> -- Thanks!
The plugins are not opened for this function so of course they
won't set their build hashes.
Fixes errors like this:
`uefi should call fu_plugin_set_build_hash()`
For composite devices you might want to restrict the child device in respect
to the parent, for instance requiring the parent to have greater than a specific
firmware version number.
The other useful thing to use this for is checking if the parent has a specific
GUID (of any version) which allows us to match against the common VID&PID
instance IDs. This would allow us to restrict a generic child device update to
a specific OEM vendor parent.
This is specified as <firmware depth="1" ...> to match the parent device and
<firmware depth="2" ...> to match the grandparent device.
This fixes a regression introduced by 2031ce3bf6
that leads to:
```
USB error on device 2dc8:5750 : No such device (it may have been disconnected) [-4]
```
The root of this problem is that we were trying to use pending.db as both a
pending report store and also a record of history, so you could see updates for
device foo 1.2.3->1.2.4 and then 1.2.4->1.2.5.
This sort-of half worked, but it meant in some cases we matched on the old
device version, sometimes on the new release version and sometimes matching on
either.
The problem was that in various places we were using the device ID as *the*
primary key, and so we had various functions like fu_history_get_device_by_id()
that expected to return just one device, not several.
The device ID also changes if the device is moved from USB plug to another, and
so it got even more complicated again trying to de-dupe the devices.
The sanest thing to do is to decide that pending.db only contains the latest
attempt to go from one version to another using device-id as the primary key.
This makes reporting work reliably now, although has the effect you can't
report a 1.2.3->1.2.4 and 1.2.4->1.2.5 at the same time. Note, if you tried to
do the latter before, the update-error (if set) would be wrong on the second
report...
For the 1.3.3 release make it all simpler and so the reporting works reliably.
Longer term we can design a better database schema that uses unique ID to
represent the *transaction* itself, that isn't tied to the device ID in any way.
This might mean extending the DBus API to cope with multiple devices being
returned with the same device-id set.
It's not usual to open the device from inside the device, otherwise there is a
possibility of a deadlock. For devices without the ->prepare() virtual function
the ->open() should either be a NOP or very fast indeed.
To debug flashing failures it's sometimes requried to get a SPI dump of the
hardware to analysis.
Add a debug-only command that lets us dump the device from the engine.
During startup we do 1898 persistent allocations to load the quirk files, which
equates to ~90kb of RSS. Use libxmlb to create a mmap'able store we can query
with XPath queries at runtime.
This makes the daemon less destructive at startup, especially if the ESP
is not mounted.
It's stored in 3 different places right now, so move it into one point of truth.
Now the ESP is detected when needed including all point of time safety checks and
dynamically mounted and unmounted if necessary.
As the plugin is allowed to change print out the entire FuDevice before
performing each step when in verbose mode.
Note, these are only show during update if run with --verbose, not at startup.
```
Uploading firmware reports helps hardware vendors to quickly identify failing and successful updates on real devices.
Upload report now? (Requires internet connection):
0. Do not upload reports at this time, but prompt again for future updates
1. Do not upload reports, and never ask to upload reports for future updates
2. Upload reports just this one time, but prompt again for future updates
3. Upload reports this time and automatically upload reports after completing future updates
1
key ReportUri not supported
```
Somewhat embarrassingly we were not actually showing the 8bitdo manual detach
images because... we were not actually exporting them. I don't know how this
ever worked in GNOME Software, but it would also explain the low 'success' rate
of the 8bitdo firmware on the LVFS.
This will also be used by Logitech in the future for the C1 Unifying devices.
To work in gnome-softare, this also needs the matching functionality to call
gs_app_add_screenshot() with the new exported data.
We don't need to enumerate USB and UDev devices in the self tests. In the case
where the probe fails (due to a permissions error, as not running as root) we
don't want the tests to fail.
When the LVFS switches over to outputting <issues> rather than appending to the
update description we need to be in a position to display the new data.
This allows plugins to rescan hardware based on uevents of any device class
registered with fu_plugin_add_udev_subsystem().
Additionally, the events are rate limited to avoid causing lots of extra plugin
processing when replugging hardware.
Users are getting confused when they're being told that AC power is required
when they are already on AC power... but the battery is too low to perform the
update.
This leads to madness, as some formats are supersets of the detected types,
e.g. 'intel-me' is detected as 'quad' and 'bcd' is detected as 'pair'.
Where the version format is defined in a specification or hardcoded in the
source use a hardcoded enum value, otherwise use a quirk override.
Additionally, warn if the version does not match the defined version format
Set `UpdateError` in this case, and prevent updates.
If the `VersionFormat` is not known, then the device will later have
problems installing firmware with errors like:
```
failed to get release version: version format unset and version 13000001 ambiguous
```
or
```
Firmware version formats were different, device was '(null)' and release is 'plain'
```
This generally should be corrected by quirking the device.
I'm getting a bit fed up with failed reports from arch users where they just
have a broken system. I don't think it's useful to upload to the LVFS or notify
the vendor about failures like this.
This allows several things, for instance:
* Adding or removing blacklisted plugins or devices
* Changing the idle timeout where allowed
...without a user needing to manually modify a configuration file.
This is arguably the more correct fix, regardless of the UEFI quirk use as each
release was reading and converting the exact same property from the component.
Fixes https://github.com/hughsie/fwupd/issues/1156
This allows us to print the daemon-specific properties like VersionFormat when
debugging. The caller can continue to operate on the subclassed FwupdDevice
without changes.
Some firmwares only update one part of the system, e.g. the EC or ME firmware.
Other updates include all the updates needed for the whole system, and vendors
have been doing different things with the component name due to this.
To fix, add an enumerated set of firmware 'categories' that can be set by the
uploader in the metainfo.xml file (or changed the LVFS) which automatically
set the name suffix.
Only append the translated version in the client when <categories> has
not been set, as the LVFS is still operating in compatibility mode and setting
the <name> with the prefix. Add the support to fwupd now so we can switch in
about 9 months time.
This stops us scheduling an update that has FWUPD_DEVICE_FLAG_REQUIRE_AC when
on battery power, as we don't show messages when in the offline updates mode.
Of course, we still check for AC when we reboot, but the common case is where
we reboot into the offline mode just after we schedule the update without
changing the hardware configuration.
There are a lot of failures in the LVFS with 'device version not updated on
success' and this is because we were looking at the original FuDevice, rather
than the FuDevice that came back after the bootloader replug phase.
Before doing the check get the new FuDevice to prevent failing the check with
a false positive.
This ensures the certificate is always present even before the user has
generated a report or manually signed test data.
Also, if the GnuTLS version is too old just log a message and continue.
For my future self, to debug the fwupd.shutdown activation failure, you can do:
mount /dev/sdc1 /mnt
/usr/libexec/fwupd/fwupdtool activate --verbose &> /mnt/log.txt
umount /dev/sdc1
...where /dev/sdc1 is an attached FAT32-formatted USB drive.
This allows us to one day implement 'reinstall', allows us to have a more useful
`get-releases` command and also means we can add other reasons for blocking the
release in the future.
In the future we'll want to use this flag to signify if the release is an
upgrade, downgrade, below the version-lowest, or if it is locked in some way.
This is intended for devices that it is not safe to immediately activate
the firmware. It may be called at a more convenient time instead.
Both fwupdmgr and fwupdtool support the feature.
- if called at runtime with fwupdmgr it uses the daemon
- during shutdown fwupdtool uses the pending.db to perform this feature.
This fixes a number of bugs, where the plugin changing was not always detected
properly. It also means we always record a error failure if _install_blob()
returns FALSE.
The idea is that if the user should know something about the device update
"after" it's succesfully completed then the plugin can set `UpdateMessage`
for the device and a client can show it.
An example would be a device that doesn't reboot on its own and the user
needs to power cycle it manually.
The source URL allows us to comply with our various obligations when shipping
firmware built from GPL licensed sources. The details URL allows vendors to
include a link to a full HTML details page about the specific release.
If the device is rebooted into a different shape, the plugin managing
the device may also be different.
This would be the case for plugins that just subclass the update_detach()
method, and leave to other plugins the actual required update procedure.
This will perform updates with all currently present metadata.
It is "intended" for usage with local metadata repositories referring
to local files.
fwupdtool however does also support fetching a file from the web
and if the metadata refers to the file on the web it should also work
for that.
Some firmware has a different on-device checksum to the hash of the firmware
file itself. This may be because:
* The content is not a binary file, e.g. Intel HEX or SREC
* Only part of the firmware is flashed, e.g. ignoring the bootloader section
* The device checksum is calculated using another method entirely, e.g. PCR0
It's also made complicated as there may be more than one 'correct' device
checksum in some cases, but nothing that a union query can't solve.
The LVFS has always sent components with the old-style '<metadata>' section
rather than the new-style '<custom>' section, and when using appstream-glib
we just accepted either element name. Although the LVFS has just been fixed to
use the new name, old metadata might be present on the system already.
When all clients have downloaded new metadata we can revert this, perhaps in a
couple of weeks time. Until then, the belt-and-braces approach is probably a
good idea.
Fixes https://github.com/hughsie/fwupd/issues/874
This metadata key is now unnecessary, as firmwares are expected to set the
version format in the metadata.
If the metadata is missing, the LVFS allows a per-vendor default for non-semver
release versions which is now unconditionally set in metadata.
Plugins are allowed to 'opt-out' of this behaviour using _RULE_INHIBITS_IDLE.
This should be used where waking up the hardware to coldplug is expensive,
either from a power consumption point of view, or if other artifacts are going
to be seem -- for instance if the screen flickers when probing display devices.
This functionality is also inhibited when the actual upgrade is happening,
for obvious reasons.
Admins can turn off this auto-sleep behaviour by editing the daemon.conf file.
Fixes https://github.com/hughsie/fwupd/issues/417
This speeds up matching for GUIDs by about 90%, taking the query from 3.17ms to
about 0.33ms on my Thinkpad. This is more important for slow ARM hardware,
where strcmp() is more expensive than on x64.
This means we don't have to have all the thousands of GUIDs as tiny allocated
strings being passed around to all plugins in a GPtrArray.
This also reduces our heap usage by about 6%.
These are set from the AppStream metadata and are specific to the firmware
release.
If not provided, the install duration falls back to the per-device duration
values which can be set in the quirk files.
If the FuDevice is changed during a replug then the `UPDATE_STATE_SUCCESS` will
not be stored on the current FuDevice, which means we store an update-state of
`unknown` to the history database.
Which means we get no success or failure reports, which is bad.
This means we are storing the distro that was used to perform the update,
rather than the distro that was used to do the upload.
Fixes https://github.com/hughsie/fwupd/issues/838
We want the XML to look like:
<custom>
<value key="fwupd::RemoteId">lvfs</value>
</custom>
not:
<custom>
<fwupd::RemoteId>lvfs</fwupd::RemoteId>
</custom>
Also, fix the XPath query string to actually match the container checksum, and
actually save the custom node to the builder source.
This fixes a regression in 1.2.0 where no reports could be uploaded.
The libxmlb library is much faster to query, and does not require the daemon
to parse the XML metadata at startup. It's a zero-copy mmap design that is more
modern and less clunky.
RSS has reduced from 3Mb (peak 3.61Mb) to 1Mb (peak 1.07Mb) and the startup
time has gone from 280ms to 250ms.
At the moment firmware can declare that it has certain requirements, for
instance the existing firmware version, the bootloader version, or the
vendor ID of the device.
In some situations we actually want to check the firmware version of a
*different* device. Good examples here would be only allowing an EC upgrade if
a specific BIOS update has already been done, or only allowing a wireless
reciever to be updated if the attached wireless devices have been updated first.
To use this, you can do something like:
<requires>
<id compare="ge" version="1.1.3">org.freedesktop.fwupd</id>
<firmware compare="ge" version="0.9.0">guid-of-other-device</firmware>
</requires>
The profiling data is of limited use, and better data can be obtained using
kcachegrind and massif. Additionally, the profile samples were the cause of the
small RSS growth over time, when in reality the data would only be shown when
the verbose switch is used at daemon startup.
Now devices can have multiple GUIDs and do not share platform IDs there is
really no point waiting to add the device.
This allows us to remove a whole lot of code for this now-unused functionality.
It wasn't hugely clear what the platform ID was actually meant to represent. In
some cases it was being used like a physical ID, in others it was a logical ID,
and in others it was both. In some cases it was even used as a sysfs path.
Clear up all the confusion by splitting the platform ID into two parts, an
optional *physical* ID to represent the electrical connection, and an optional
*logical* ID to disambiguate composite devices with the same physical ID.
Also create an explicit sysfs_path getter for FuUdevDevice to make this clear.
This allows WAIT_FOR_REPLUG to always work, rather than depending on the order
that the GUIDs were added, and that the kernel would always return the same
sysfs path (which it doesn't have to do, especially for hidraw devices).
The daemon creates a baseclass of either FuUsbDevice or FuUdevDevice when the
devices are added or coldplugged to match the quirk database and to find out
what plugin to run.
This is proxied to plugins, but they are given the GUsbDevice or GUdevDevice and
the FuDevice is just thrown away. Most plugins either use a FuUsbDevice or
superclassed version like FuNvmeDevice and so we re-create the FuDevice, re-probe
the hardware, re-query the quirk database and then return this to the daemon.
In some cases, plugins actually probe the hardware three times (!) by creating
a FuUsbDevice to get the quirks, so that the plugin knows what kind of
superclass to create, which then itself probes the hardware again.
Passing the temporary FuDevice to the plugins means that the simplest ones can
just fu_plugin_device_add() the passed in object, or create a superclass and
incorporate the actual GUsbDevice and all the GUIDs.
This breaks internal plugin API but speeds up startup substantially and deletes
a lot of code.
Similar to commit 7d35278 regarding the install option with a daemon compiled without
GPG, but now, if you also have a daemon compiled without GPG, and if you have downloaded a
CAB file from LVFS and ran get-details, it would fail to display the contents
of the associated CAB file.
This allows it to still display the contents of the CAB file, and
a warning will be displayed in the logs.
This allows us to match non-DeviceID GUIDs, and also GUIDs we don't know how to
generate.
To make this fully useful, search for device quirks when GUIDs are added.
Allowing plugins to 'register' for different udev subsystems in _init() allows
us to move the client to the engine, reducing the number of wakeups considerably
for each device event. It also will reduce the amount of boilerplate code in
each plugin that uses GUdev, much like we have done for GUsb.
Some devices take a long time to initialize and tools like fu-tool
have no feedback during this time.
Show a status message to indicate what's going on.
This isn't useful for most people and just takes up space in the logs.
When there is a problem it's typically specifically with a plugin
and at that time we ask them to run the daemon with verbose anyway.
In this instance, we define the 'same device' to be a FuDevice that has at
least one matching GUID. We allow the plugins to define which one is 'better'
than other plugins, and use this to only have one FuDevice for the physical
device.
Alternative to https://github.com/hughsie/fwupd/pull/604
Fix a logic error when testing requirements; what was supposed to happen was
that a the firmware requirements were only processed when we had a FuDevice set
in the FuInstallTask, but instead we were falling through to the handler for
'unknown requirement type'.
