As the plugin is allowed to change print out the entire FuDevice before
performing each step when in verbose mode.
Note, these are only show during update if run with --verbose, not at startup.
```
Uploading firmware reports helps hardware vendors to quickly identify failing and successful updates on real devices.
Upload report now? (Requires internet connection):
0. Do not upload reports at this time, but prompt again for future updates
1. Do not upload reports, and never ask to upload reports for future updates
2. Upload reports just this one time, but prompt again for future updates
3. Upload reports this time and automatically upload reports after completing future updates
1
key ReportUri not supported
```
Somewhat embarrassingly we were not actually showing the 8bitdo manual detach
images because... we were not actually exporting them. I don't know how this
ever worked in GNOME Software, but it would also explain the low 'success' rate
of the 8bitdo firmware on the LVFS.
This will also be used by Logitech in the future for the C1 Unifying devices.
To work in gnome-softare, this also needs the matching functionality to call
gs_app_add_screenshot() with the new exported data.
We don't need to enumerate USB and UDev devices in the self tests. In the case
where the probe fails (due to a permissions error, as not running as root) we
don't want the tests to fail.
When the LVFS switches over to outputting <issues> rather than appending to the
update description we need to be in a position to display the new data.
This allows plugins to rescan hardware based on uevents of any device class
registered with fu_plugin_add_udev_subsystem().
Additionally, the events are rate limited to avoid causing lots of extra plugin
processing when replugging hardware.
Users are getting confused when they're being told that AC power is required
when they are already on AC power... but the battery is too low to perform the
update.
This leads to madness, as some formats are supersets of the detected types,
e.g. 'intel-me' is detected as 'quad' and 'bcd' is detected as 'pair'.
Where the version format is defined in a specification or hardcoded in the
source use a hardcoded enum value, otherwise use a quirk override.
Additionally, warn if the version does not match the defined version format
Set `UpdateError` in this case, and prevent updates.
If the `VersionFormat` is not known, then the device will later have
problems installing firmware with errors like:
```
failed to get release version: version format unset and version 13000001 ambiguous
```
or
```
Firmware version formats were different, device was '(null)' and release is 'plain'
```
This generally should be corrected by quirking the device.
I'm getting a bit fed up with failed reports from arch users where they just
have a broken system. I don't think it's useful to upload to the LVFS or notify
the vendor about failures like this.
This allows several things, for instance:
* Adding or removing blacklisted plugins or devices
* Changing the idle timeout where allowed
...without a user needing to manually modify a configuration file.
This is arguably the more correct fix, regardless of the UEFI quirk use as each
release was reading and converting the exact same property from the component.
Fixes https://github.com/hughsie/fwupd/issues/1156
This allows us to print the daemon-specific properties like VersionFormat when
debugging. The caller can continue to operate on the subclassed FwupdDevice
without changes.
Some firmwares only update one part of the system, e.g. the EC or ME firmware.
Other updates include all the updates needed for the whole system, and vendors
have been doing different things with the component name due to this.
To fix, add an enumerated set of firmware 'categories' that can be set by the
uploader in the metainfo.xml file (or changed the LVFS) which automatically
set the name suffix.
Only append the translated version in the client when <categories> has
not been set, as the LVFS is still operating in compatibility mode and setting
the <name> with the prefix. Add the support to fwupd now so we can switch in
about 9 months time.
This stops us scheduling an update that has FWUPD_DEVICE_FLAG_REQUIRE_AC when
on battery power, as we don't show messages when in the offline updates mode.
Of course, we still check for AC when we reboot, but the common case is where
we reboot into the offline mode just after we schedule the update without
changing the hardware configuration.
There are a lot of failures in the LVFS with 'device version not updated on
success' and this is because we were looking at the original FuDevice, rather
than the FuDevice that came back after the bootloader replug phase.
Before doing the check get the new FuDevice to prevent failing the check with
a false positive.
This ensures the certificate is always present even before the user has
generated a report or manually signed test data.
Also, if the GnuTLS version is too old just log a message and continue.
For my future self, to debug the fwupd.shutdown activation failure, you can do:
mount /dev/sdc1 /mnt
/usr/libexec/fwupd/fwupdtool activate --verbose &> /mnt/log.txt
umount /dev/sdc1
...where /dev/sdc1 is an attached FAT32-formatted USB drive.
This allows us to one day implement 'reinstall', allows us to have a more useful
`get-releases` command and also means we can add other reasons for blocking the
release in the future.
In the future we'll want to use this flag to signify if the release is an
upgrade, downgrade, below the version-lowest, or if it is locked in some way.
This is intended for devices that it is not safe to immediately activate
the firmware. It may be called at a more convenient time instead.
Both fwupdmgr and fwupdtool support the feature.
- if called at runtime with fwupdmgr it uses the daemon
- during shutdown fwupdtool uses the pending.db to perform this feature.
This fixes a number of bugs, where the plugin changing was not always detected
properly. It also means we always record a error failure if _install_blob()
returns FALSE.
The idea is that if the user should know something about the device update
"after" it's succesfully completed then the plugin can set `UpdateMessage`
for the device and a client can show it.
An example would be a device that doesn't reboot on its own and the user
needs to power cycle it manually.
The source URL allows us to comply with our various obligations when shipping
firmware built from GPL licensed sources. The details URL allows vendors to
include a link to a full HTML details page about the specific release.
If the device is rebooted into a different shape, the plugin managing
the device may also be different.
This would be the case for plugins that just subclass the update_detach()
method, and leave to other plugins the actual required update procedure.
This will perform updates with all currently present metadata.
It is "intended" for usage with local metadata repositories referring
to local files.
fwupdtool however does also support fetching a file from the web
and if the metadata refers to the file on the web it should also work
for that.
Some firmware has a different on-device checksum to the hash of the firmware
file itself. This may be because:
* The content is not a binary file, e.g. Intel HEX or SREC
* Only part of the firmware is flashed, e.g. ignoring the bootloader section
* The device checksum is calculated using another method entirely, e.g. PCR0
It's also made complicated as there may be more than one 'correct' device
checksum in some cases, but nothing that a union query can't solve.
The LVFS has always sent components with the old-style '<metadata>' section
rather than the new-style '<custom>' section, and when using appstream-glib
we just accepted either element name. Although the LVFS has just been fixed to
use the new name, old metadata might be present on the system already.
When all clients have downloaded new metadata we can revert this, perhaps in a
couple of weeks time. Until then, the belt-and-braces approach is probably a
good idea.
Fixes https://github.com/hughsie/fwupd/issues/874
This metadata key is now unnecessary, as firmwares are expected to set the
version format in the metadata.
If the metadata is missing, the LVFS allows a per-vendor default for non-semver
release versions which is now unconditionally set in metadata.
Plugins are allowed to 'opt-out' of this behaviour using _RULE_INHIBITS_IDLE.
This should be used where waking up the hardware to coldplug is expensive,
either from a power consumption point of view, or if other artifacts are going
to be seem -- for instance if the screen flickers when probing display devices.
This functionality is also inhibited when the actual upgrade is happening,
for obvious reasons.
Admins can turn off this auto-sleep behaviour by editing the daemon.conf file.
Fixes https://github.com/hughsie/fwupd/issues/417
This speeds up matching for GUIDs by about 90%, taking the query from 3.17ms to
about 0.33ms on my Thinkpad. This is more important for slow ARM hardware,
where strcmp() is more expensive than on x64.
This means we don't have to have all the thousands of GUIDs as tiny allocated
strings being passed around to all plugins in a GPtrArray.
This also reduces our heap usage by about 6%.
These are set from the AppStream metadata and are specific to the firmware
release.
If not provided, the install duration falls back to the per-device duration
values which can be set in the quirk files.
If the FuDevice is changed during a replug then the `UPDATE_STATE_SUCCESS` will
not be stored on the current FuDevice, which means we store an update-state of
`unknown` to the history database.
Which means we get no success or failure reports, which is bad.
This means we are storing the distro that was used to perform the update,
rather than the distro that was used to do the upload.
Fixes https://github.com/hughsie/fwupd/issues/838
We want the XML to look like:
<custom>
<value key="fwupd::RemoteId">lvfs</value>
</custom>
not:
<custom>
<fwupd::RemoteId>lvfs</fwupd::RemoteId>
</custom>
Also, fix the XPath query string to actually match the container checksum, and
actually save the custom node to the builder source.
This fixes a regression in 1.2.0 where no reports could be uploaded.
The libxmlb library is much faster to query, and does not require the daemon
to parse the XML metadata at startup. It's a zero-copy mmap design that is more
modern and less clunky.
RSS has reduced from 3Mb (peak 3.61Mb) to 1Mb (peak 1.07Mb) and the startup
time has gone from 280ms to 250ms.
At the moment firmware can declare that it has certain requirements, for
instance the existing firmware version, the bootloader version, or the
vendor ID of the device.
In some situations we actually want to check the firmware version of a
*different* device. Good examples here would be only allowing an EC upgrade if
a specific BIOS update has already been done, or only allowing a wireless
reciever to be updated if the attached wireless devices have been updated first.
To use this, you can do something like:
<requires>
<id compare="ge" version="1.1.3">org.freedesktop.fwupd</id>
<firmware compare="ge" version="0.9.0">guid-of-other-device</firmware>
</requires>
The profiling data is of limited use, and better data can be obtained using
kcachegrind and massif. Additionally, the profile samples were the cause of the
small RSS growth over time, when in reality the data would only be shown when
the verbose switch is used at daemon startup.
Now devices can have multiple GUIDs and do not share platform IDs there is
really no point waiting to add the device.
This allows us to remove a whole lot of code for this now-unused functionality.
It wasn't hugely clear what the platform ID was actually meant to represent. In
some cases it was being used like a physical ID, in others it was a logical ID,
and in others it was both. In some cases it was even used as a sysfs path.
Clear up all the confusion by splitting the platform ID into two parts, an
optional *physical* ID to represent the electrical connection, and an optional
*logical* ID to disambiguate composite devices with the same physical ID.
Also create an explicit sysfs_path getter for FuUdevDevice to make this clear.
This allows WAIT_FOR_REPLUG to always work, rather than depending on the order
that the GUIDs were added, and that the kernel would always return the same
sysfs path (which it doesn't have to do, especially for hidraw devices).
The daemon creates a baseclass of either FuUsbDevice or FuUdevDevice when the
devices are added or coldplugged to match the quirk database and to find out
what plugin to run.
This is proxied to plugins, but they are given the GUsbDevice or GUdevDevice and
the FuDevice is just thrown away. Most plugins either use a FuUsbDevice or
superclassed version like FuNvmeDevice and so we re-create the FuDevice, re-probe
the hardware, re-query the quirk database and then return this to the daemon.
In some cases, plugins actually probe the hardware three times (!) by creating
a FuUsbDevice to get the quirks, so that the plugin knows what kind of
superclass to create, which then itself probes the hardware again.
Passing the temporary FuDevice to the plugins means that the simplest ones can
just fu_plugin_device_add() the passed in object, or create a superclass and
incorporate the actual GUsbDevice and all the GUIDs.
This breaks internal plugin API but speeds up startup substantially and deletes
a lot of code.
Similar to commit 7d35278 regarding the install option with a daemon compiled without
GPG, but now, if you also have a daemon compiled without GPG, and if you have downloaded a
CAB file from LVFS and ran get-details, it would fail to display the contents
of the associated CAB file.
This allows it to still display the contents of the CAB file, and
a warning will be displayed in the logs.
This allows us to match non-DeviceID GUIDs, and also GUIDs we don't know how to
generate.
To make this fully useful, search for device quirks when GUIDs are added.
Allowing plugins to 'register' for different udev subsystems in _init() allows
us to move the client to the engine, reducing the number of wakeups considerably
for each device event. It also will reduce the amount of boilerplate code in
each plugin that uses GUdev, much like we have done for GUsb.
Some devices take a long time to initialize and tools like fu-tool
have no feedback during this time.
Show a status message to indicate what's going on.
This isn't useful for most people and just takes up space in the logs.
When there is a problem it's typically specifically with a plugin
and at that time we ask them to run the daemon with verbose anyway.
In this instance, we define the 'same device' to be a FuDevice that has at
least one matching GUID. We allow the plugins to define which one is 'better'
than other plugins, and use this to only have one FuDevice for the physical
device.
Alternative to https://github.com/hughsie/fwupd/pull/604
Fix a logic error when testing requirements; what was supposed to happen was
that a the firmware requirements were only processed when we had a FuDevice set
in the FuInstallTask, but instead we were falling through to the handler for
'unknown requirement type'.
This pivots the data storage so that the group is used as the preconditon
and the key name is used as the parameter to change. This allows a more natural
data flow, where a new device needs one new group and a few few keys, rather
than multiple groups, each with one key.
This also allows us to remove the key globbing when matching the version format
which is often a source of confusion.
Whilst changing all the quirk files, change the key prefixes to be more familiar
to Windows users (e.g. Hwid -> Smbios, and FuUsbDevice -> DeviceInstanceId)
who have to use the same IDs in Windows Update.
This also allows us to pre-match the desired plugin, rather than calling the
probe() function on each plugin.
Not calling dozens of ->probe() functions is dramatically more efficient when
there are a large number of USB devices attached.
This requires plugins to 'opt-in' to the new behaviour, and also to supply
quirks that match all devices.
If the daemon either de-duplicates or replaces the object passed emitted from
device-added then the object set as the alternate may not be the same instance
as the daemon version. This causes weird things to happen.
To make this less fragile, specify the *ID* of the object that should be the
alternate device, which allows the daemon to do clever things, and then assign
the object from the ID as the last step.
Although fixing no bug, this makes implementing future functionality easier.
Now we can update multiple devices (in multiple plugins) using one firmware
archive we need a way to cleanup after all the plugins have been run.
Fixes https://github.com/hughsie/fwupd/issues/561
This can be used to select the AA.BB.CC.DD format rather than the default
AA.BB.CCDD format for firmware versions. This allows us to support new vendor
requirements without adding more complicated rules to the quirk file.
Fixes a segfault that occurs during cleanup of USB plugins.
When g_module_close was called memory allocated by the plugin would
get freed leading to the finalize method for object class pointing
to garbage.
When developing code it's really convenient to only run the new plugin. This
means you don't have to wait for the other hardware to initialize and there
are no side-effects from other plugins when installing firmware.
You can specify multiple plugins as globs, for instance:
fwupdtool get-devices \
--plugin-whitelist wacom \
--plugin-whitelist "thunderbolt*"
When using failed to open firmware.cab we pass in a device ID of '*' which
tells the daemon to update anything that matches. The current implementation
will fail in two ways:
* If duplicate hardware is installed (for instance two Unifying receivers) then
only the first matching device will be updated.
* If the firmware archive contains two different images then we only try and
upgrade the first device that matches. This means we're unable to update
composite devices using one firmware file.
To fix both issues, carefully build a list of tasks that can be processed using
the given firmware and installed devices, request authentication using all the
different action IDs, then upgrade all the devices one-at-a-time.
Based on a patch by Mario Limonciello <mario.limonciello@dell.com>, many thanks.
This allows us to find out the logical parent device, for instance in composite
devices with more than one firmware image for a single device.
We also allow lazily specifying the device parent using a GUID and the engine
then automatically sets the parent object when the GUIDs match, which allows
children and parents to exist in different plugins.
We don't want to use the version= attribute as the HWIDs are not versions, and
thinkgs like globbing just doesn't make sense given they are from hashes.
Fixes the client side part of https://github.com/hughsie/lvfs-website/issues/110
This allows plugins to set and explicit build-time version. It also uses the
same AppStream component-ID scheme rather than the home-grown 'FooVersion' key.
Also, use the new runtime and compile-time versions in the report metadata.
Due to the key change we'll also need to update some LVFS rules.
In some cases firmware can only be installed with an up to date GUsb (e.g. with
some STM-DFU hardware) or with a new version of fwupdate (e.g. any UEFI
UpdateCapsule without a capsule header).
We should be able to match against other software versions like we can the
fwupd version, e.g.
<requires>
<id compare="ge" version="0.9.2">org.freedesktop.fwupd</id>
<id compare="ge" version="11">com.redhat.fwupdate</id>
</requires>
Also, rather than checking each requirement we know about on the component,
check each requirement on the component about things we know. This ensures we
don't allow firmware to be installs that requires for instance fwupdate 22 when
the runtime version is only being added in fwupdate 12 and up.
This means the following is now an error that will fail to allow the firmware
to be installed:
<requires>
<firmware>doesnotexist</firmware>
<some_future_tag>also_unknown</some_future_tag>
</requires>
Also add a lot of self tests to test the various new failure modes.
Fixes https://github.com/hughsie/fwupd/issues/463
There are a lot of hacks here;
* Pulling newer libappstream-glib from Fedora
* Pulling a systemd backport
* Manually installing pillow and pygobject
* PKCS7 is turned off (gnutls is too old)
Don't optimize the device flag away. We need to be able to set the flag on all
devices that match the device ID, not just the one that happens to match now.
If the device firmware was set incorrectly make then set it to the release
version so the database update works correctly. We can't do any kind of vercmp
in the database, so use a daemon warning so we can either fix the plugin or
the XML.
This fixes up the issue that the hardware reports '28.00' and the AppStream
release specifies '28.0'.
Fixes: https://github.com/hughsie/fwupd/issues/387
If this happens:
1.2.3 -> 1.2.4 = failure
1.2.3 -> 1.2.5 = success
...we want to preserve both in the history database so they can both be shared
with the LVFS. Use the device ID and the new and old firmware versions when
modifying and deleting entries.
This fix is made slightly more tricky as we have to drop the PRIMARY KEY
attribute on the device_id, and due to a limitation in SQLite, it means copying
the old history into a new table.
Previous to having the history database we could only notify about firmware that
as installed using the uefi plugin, as that had a few system-wide API calls to
say 'this update failed' or 'this was the error'.
Now we have the local history database not only can we report more details about
the UEFI update (e.g. the old version number) but we can also offer the same
functionality for all other plugins.
Although this does rework how the data for GetResults() is populated, it does
make the FuEngine object quite a lot less confused.
It also fixes a warning in the fwupd plugin for gnome-software, which was
expecting the FwupdRelease to be populated for the FwupdDevice.
This allows us to record whether we've shown the user a notification (either in
the terminal or in a GUI) that an update failed or was successful.
This can't be done in the session otherwise we'd get a notification for every
different user on the system. Notifying also isn't the same as reporting,
although one can certainly follow on from the latter.
In the case of failing to even set up UpdateCapsule, the uefi plugin would
dutifully return SUCCESS as it was referring to the 2nd-to-last update that
actually worked.
The SUPPORTED flag is used when a device appears in the AppStream metadata of
any enabled remote, so when we rescan the modified store also ensure the flag
state is still correct.
Fixes https://github.com/hughsie/fwupd/issues/363
Rename FuPending to FuHistory to better represent what the object is now doing.
Also, while we're here, switch to using SQLite prepared statements to avoid a
possible invalid read on i386 hardware.
Always set the AppStream app properties on the FwupdRelease. In some cases we
were returning FwupdRelease objects with no name or summary which gnome-software
was ignoring.
Using old versions of gcab we could only do one thing: extract the files in the
cabinet archive to a new directory in /tmp, and then fwupd would have to read
them back in to memory to parse them. This was both inelegant and wasteful, and
probably not an awesome idea from a security or privacy point of view.
Using libgcab >= 1.0 we can decompress to a GBytes blob, and then verify the
firmware and metainfo file without anything being written to disk.
As this is a security sensitive operation, move the fwupd-specific helper code
out of libappstream-glib and also add a lot of internal self tests.
The gcab code will have to remain in libappstream-glib for a long time, but we
don't have to use it. Handling the cab file here also allows us to fix two
long-standing bugs:
* MetaInfo or firmware files in a subdirectory are handled correctly
* The archive can also be self-signed using PKCS7 instead of using GPG
In the case of multiple <component> sections with different AppStream IDs, but
with the same GUID <provides>, filter using the requirements rather than just
choosing the first one.
This allows the update of Logitech devices with secure bootloaders. Many thanks
to Ogier Bouvier for identifying the problem.
This ensures we get progress events when replugging a device. Also, remove the
callbacks on the 'old' device to avoid causing multiple events on a 2nd-replug.
This makes more sense; we're updating the device, not the plugin itself.
This also means we don't need to funnel everything through callbacks like
GFileProgressCallback and we can also update the state without adding an
explicit callback to each derived device type.
`Not compatible with fwupd version 1.0.2, requires >= 1.0.3`
...is easier to understand than...
`Value of org.freedesktop.fwupd incorrect: failed predicate [1.0.3 ge 1.0.2]`
This saves all the USB plugins from connecting to the context and managing the
device lifecycle and allows devices that uses FuUsbDevice to be removed
automatically.
This makes supported plugins *much* smaller indeed.
When changing from runtime->bootloader->runtime the usual way of handling this
in a fwupd plugin is to:
* reset the device and wait for a replug
* flash the hardware
* reset the device and wait for a replug
This works well when the runtime and bootloader modes are handled by the same
plugin. For situations like the Nitrokey device, where one plugin handles the
runtime (nitrokey), and another handles the bootloader (dfu) we have to have
the ability to 'ignore' the device removal and just issue a 'changed' signal
so the client refreshes the properties.
This moves more functionality out of the engine, and will allow us to add some
cleverness to the device list to allow the FuDevice to be shared between
different plugins.
It's actually less scary to see a SHA1 hash than it is to see a path like
/sys/devices/pci0000:00/0000:00:1d.0/usb1/1-1/1-1.2. It's also way easier to
copy and paste into the various fwupdmgr command that require a device ID and
also means we can match a partial prefix much like git allows.
If we also move to a model where plugins can be changed during different stages
of the update (e.g. during detach) then the device might change connection type
and then the sysfs path not only becomes difficult to paste, but incorrect.
Session software doesn't care about the format of the device ID (it is supposed
to be an implementation detail) and so there's no API or ABI break here. A few
plugins also needed to be ported, but nothing too worrying.
This allows us to do two things:
* Attach after a failed update, so the user isn't left with 'dead' hardware
* Split the detach and attach actions into different plugins in the future
This also allows us to have a separate vfunc to get the new version number
after flashing the firmware, as this may be handled in a different plugin to
the detach phase.
In the latest version of the LVFS you can restrict the firmware to a specific
machine type, for instance a specific baseboard vendor. This is the same as
done in Microsoft Update using the CHID mechanism.
This commit adds support for the <hardware> requires type, although it needs to
be built against appstream-glib 0.7.4 to be supported and/or tested.
When fwupd is installed in long-term support distros it's very hard to backport
new versions as new hardware is released.
There are several reasons why we can't just include the mapping and quirk
information in the AppStream metadata:
* The extra data is hugely specific to the installed fwupd plugin versions
* The device-id is per-device, and the mapping is usually per-plugin
* Often the information is needed before the FuDevice is created
* There are security implications in allowing plugins to handle new devices
The idea with quirks is that the end user can drop an additional (or replace
an existing) file in a .d director with a simple format and the hardware will
magically start working. This assumes no new quirks are required, as this would
obviously need code changes, but allows us to get most existing devices working
in an easy way without the user compiling anything.
This allows us to fix issues like https://github.com/hughsie/fwupd/issues/265
