There are a lot of failures in the LVFS with 'device version not updated on
success' and this is because we were looking at the original FuDevice, rather
than the FuDevice that came back after the bootloader replug phase.
Before doing the check get the new FuDevice to prevent failing the check with
a false positive.
This ensures the certificate is always present even before the user has
generated a report or manually signed test data.
Also, if the GnuTLS version is too old just log a message and continue.
For my future self, to debug the fwupd.shutdown activation failure, you can do:
mount /dev/sdc1 /mnt
/usr/libexec/fwupd/fwupdtool activate --verbose &> /mnt/log.txt
umount /dev/sdc1
...where /dev/sdc1 is an attached FAT32-formatted USB drive.
This allows us to one day implement 'reinstall', allows us to have a more useful
`get-releases` command and also means we can add other reasons for blocking the
release in the future.
In the future we'll want to use this flag to signify if the release is an
upgrade, downgrade, below the version-lowest, or if it is locked in some way.
This is intended for devices that it is not safe to immediately activate
the firmware. It may be called at a more convenient time instead.
Both fwupdmgr and fwupdtool support the feature.
- if called at runtime with fwupdmgr it uses the daemon
- during shutdown fwupdtool uses the pending.db to perform this feature.
This fixes a number of bugs, where the plugin changing was not always detected
properly. It also means we always record a error failure if _install_blob()
returns FALSE.
The idea is that if the user should know something about the device update
"after" it's succesfully completed then the plugin can set `UpdateMessage`
for the device and a client can show it.
An example would be a device that doesn't reboot on its own and the user
needs to power cycle it manually.
The source URL allows us to comply with our various obligations when shipping
firmware built from GPL licensed sources. The details URL allows vendors to
include a link to a full HTML details page about the specific release.
If the device is rebooted into a different shape, the plugin managing
the device may also be different.
This would be the case for plugins that just subclass the update_detach()
method, and leave to other plugins the actual required update procedure.
This will perform updates with all currently present metadata.
It is "intended" for usage with local metadata repositories referring
to local files.
fwupdtool however does also support fetching a file from the web
and if the metadata refers to the file on the web it should also work
for that.
Some firmware has a different on-device checksum to the hash of the firmware
file itself. This may be because:
* The content is not a binary file, e.g. Intel HEX or SREC
* Only part of the firmware is flashed, e.g. ignoring the bootloader section
* The device checksum is calculated using another method entirely, e.g. PCR0
It's also made complicated as there may be more than one 'correct' device
checksum in some cases, but nothing that a union query can't solve.
The LVFS has always sent components with the old-style '<metadata>' section
rather than the new-style '<custom>' section, and when using appstream-glib
we just accepted either element name. Although the LVFS has just been fixed to
use the new name, old metadata might be present on the system already.
When all clients have downloaded new metadata we can revert this, perhaps in a
couple of weeks time. Until then, the belt-and-braces approach is probably a
good idea.
Fixes https://github.com/hughsie/fwupd/issues/874
This metadata key is now unnecessary, as firmwares are expected to set the
version format in the metadata.
If the metadata is missing, the LVFS allows a per-vendor default for non-semver
release versions which is now unconditionally set in metadata.
Plugins are allowed to 'opt-out' of this behaviour using _RULE_INHIBITS_IDLE.
This should be used where waking up the hardware to coldplug is expensive,
either from a power consumption point of view, or if other artifacts are going
to be seem -- for instance if the screen flickers when probing display devices.
This functionality is also inhibited when the actual upgrade is happening,
for obvious reasons.
Admins can turn off this auto-sleep behaviour by editing the daemon.conf file.
Fixes https://github.com/hughsie/fwupd/issues/417
This speeds up matching for GUIDs by about 90%, taking the query from 3.17ms to
about 0.33ms on my Thinkpad. This is more important for slow ARM hardware,
where strcmp() is more expensive than on x64.
This means we don't have to have all the thousands of GUIDs as tiny allocated
strings being passed around to all plugins in a GPtrArray.
This also reduces our heap usage by about 6%.
These are set from the AppStream metadata and are specific to the firmware
release.
If not provided, the install duration falls back to the per-device duration
values which can be set in the quirk files.
If the FuDevice is changed during a replug then the `UPDATE_STATE_SUCCESS` will
not be stored on the current FuDevice, which means we store an update-state of
`unknown` to the history database.
Which means we get no success or failure reports, which is bad.
This means we are storing the distro that was used to perform the update,
rather than the distro that was used to do the upload.
Fixes https://github.com/hughsie/fwupd/issues/838
We want the XML to look like:
<custom>
<value key="fwupd::RemoteId">lvfs</value>
</custom>
not:
<custom>
<fwupd::RemoteId>lvfs</fwupd::RemoteId>
</custom>
Also, fix the XPath query string to actually match the container checksum, and
actually save the custom node to the builder source.
This fixes a regression in 1.2.0 where no reports could be uploaded.
The libxmlb library is much faster to query, and does not require the daemon
to parse the XML metadata at startup. It's a zero-copy mmap design that is more
modern and less clunky.
RSS has reduced from 3Mb (peak 3.61Mb) to 1Mb (peak 1.07Mb) and the startup
time has gone from 280ms to 250ms.
At the moment firmware can declare that it has certain requirements, for
instance the existing firmware version, the bootloader version, or the
vendor ID of the device.
In some situations we actually want to check the firmware version of a
*different* device. Good examples here would be only allowing an EC upgrade if
a specific BIOS update has already been done, or only allowing a wireless
reciever to be updated if the attached wireless devices have been updated first.
To use this, you can do something like:
<requires>
<id compare="ge" version="1.1.3">org.freedesktop.fwupd</id>
<firmware compare="ge" version="0.9.0">guid-of-other-device</firmware>
</requires>
The profiling data is of limited use, and better data can be obtained using
kcachegrind and massif. Additionally, the profile samples were the cause of the
small RSS growth over time, when in reality the data would only be shown when
the verbose switch is used at daemon startup.
Now devices can have multiple GUIDs and do not share platform IDs there is
really no point waiting to add the device.
This allows us to remove a whole lot of code for this now-unused functionality.
It wasn't hugely clear what the platform ID was actually meant to represent. In
some cases it was being used like a physical ID, in others it was a logical ID,
and in others it was both. In some cases it was even used as a sysfs path.
Clear up all the confusion by splitting the platform ID into two parts, an
optional *physical* ID to represent the electrical connection, and an optional
*logical* ID to disambiguate composite devices with the same physical ID.
Also create an explicit sysfs_path getter for FuUdevDevice to make this clear.
This allows WAIT_FOR_REPLUG to always work, rather than depending on the order
that the GUIDs were added, and that the kernel would always return the same
sysfs path (which it doesn't have to do, especially for hidraw devices).
The daemon creates a baseclass of either FuUsbDevice or FuUdevDevice when the
devices are added or coldplugged to match the quirk database and to find out
what plugin to run.
This is proxied to plugins, but they are given the GUsbDevice or GUdevDevice and
the FuDevice is just thrown away. Most plugins either use a FuUsbDevice or
superclassed version like FuNvmeDevice and so we re-create the FuDevice, re-probe
the hardware, re-query the quirk database and then return this to the daemon.
In some cases, plugins actually probe the hardware three times (!) by creating
a FuUsbDevice to get the quirks, so that the plugin knows what kind of
superclass to create, which then itself probes the hardware again.
Passing the temporary FuDevice to the plugins means that the simplest ones can
just fu_plugin_device_add() the passed in object, or create a superclass and
incorporate the actual GUsbDevice and all the GUIDs.
This breaks internal plugin API but speeds up startup substantially and deletes
a lot of code.
Similar to commit 7d35278 regarding the install option with a daemon compiled without
GPG, but now, if you also have a daemon compiled without GPG, and if you have downloaded a
CAB file from LVFS and ran get-details, it would fail to display the contents
of the associated CAB file.
This allows it to still display the contents of the CAB file, and
a warning will be displayed in the logs.
This allows us to match non-DeviceID GUIDs, and also GUIDs we don't know how to
generate.
To make this fully useful, search for device quirks when GUIDs are added.
Allowing plugins to 'register' for different udev subsystems in _init() allows
us to move the client to the engine, reducing the number of wakeups considerably
for each device event. It also will reduce the amount of boilerplate code in
each plugin that uses GUdev, much like we have done for GUsb.
Some devices take a long time to initialize and tools like fu-tool
have no feedback during this time.
Show a status message to indicate what's going on.
This isn't useful for most people and just takes up space in the logs.
When there is a problem it's typically specifically with a plugin
and at that time we ask them to run the daemon with verbose anyway.
In this instance, we define the 'same device' to be a FuDevice that has at
least one matching GUID. We allow the plugins to define which one is 'better'
than other plugins, and use this to only have one FuDevice for the physical
device.
Alternative to https://github.com/hughsie/fwupd/pull/604
Fix a logic error when testing requirements; what was supposed to happen was
that a the firmware requirements were only processed when we had a FuDevice set
in the FuInstallTask, but instead we were falling through to the handler for
'unknown requirement type'.
This pivots the data storage so that the group is used as the preconditon
and the key name is used as the parameter to change. This allows a more natural
data flow, where a new device needs one new group and a few few keys, rather
than multiple groups, each with one key.
This also allows us to remove the key globbing when matching the version format
which is often a source of confusion.
Whilst changing all the quirk files, change the key prefixes to be more familiar
to Windows users (e.g. Hwid -> Smbios, and FuUsbDevice -> DeviceInstanceId)
who have to use the same IDs in Windows Update.
This also allows us to pre-match the desired plugin, rather than calling the
probe() function on each plugin.
Not calling dozens of ->probe() functions is dramatically more efficient when
there are a large number of USB devices attached.
This requires plugins to 'opt-in' to the new behaviour, and also to supply
quirks that match all devices.
If the daemon either de-duplicates or replaces the object passed emitted from
device-added then the object set as the alternate may not be the same instance
as the daemon version. This causes weird things to happen.
To make this less fragile, specify the *ID* of the object that should be the
alternate device, which allows the daemon to do clever things, and then assign
the object from the ID as the last step.
Although fixing no bug, this makes implementing future functionality easier.
Now we can update multiple devices (in multiple plugins) using one firmware
archive we need a way to cleanup after all the plugins have been run.
Fixes https://github.com/hughsie/fwupd/issues/561
This can be used to select the AA.BB.CC.DD format rather than the default
AA.BB.CCDD format for firmware versions. This allows us to support new vendor
requirements without adding more complicated rules to the quirk file.
Fixes a segfault that occurs during cleanup of USB plugins.
When g_module_close was called memory allocated by the plugin would
get freed leading to the finalize method for object class pointing
to garbage.
When developing code it's really convenient to only run the new plugin. This
means you don't have to wait for the other hardware to initialize and there
are no side-effects from other plugins when installing firmware.
You can specify multiple plugins as globs, for instance:
fwupdtool get-devices \
--plugin-whitelist wacom \
--plugin-whitelist "thunderbolt*"
When using failed to open firmware.cab we pass in a device ID of '*' which
tells the daemon to update anything that matches. The current implementation
will fail in two ways:
* If duplicate hardware is installed (for instance two Unifying receivers) then
only the first matching device will be updated.
* If the firmware archive contains two different images then we only try and
upgrade the first device that matches. This means we're unable to update
composite devices using one firmware file.
To fix both issues, carefully build a list of tasks that can be processed using
the given firmware and installed devices, request authentication using all the
different action IDs, then upgrade all the devices one-at-a-time.
Based on a patch by Mario Limonciello <mario.limonciello@dell.com>, many thanks.
This allows us to find out the logical parent device, for instance in composite
devices with more than one firmware image for a single device.
We also allow lazily specifying the device parent using a GUID and the engine
then automatically sets the parent object when the GUIDs match, which allows
children and parents to exist in different plugins.
We don't want to use the version= attribute as the HWIDs are not versions, and
thinkgs like globbing just doesn't make sense given they are from hashes.
Fixes the client side part of https://github.com/hughsie/lvfs-website/issues/110
This allows plugins to set and explicit build-time version. It also uses the
same AppStream component-ID scheme rather than the home-grown 'FooVersion' key.
Also, use the new runtime and compile-time versions in the report metadata.
Due to the key change we'll also need to update some LVFS rules.
In some cases firmware can only be installed with an up to date GUsb (e.g. with
some STM-DFU hardware) or with a new version of fwupdate (e.g. any UEFI
UpdateCapsule without a capsule header).
We should be able to match against other software versions like we can the
fwupd version, e.g.
<requires>
<id compare="ge" version="0.9.2">org.freedesktop.fwupd</id>
<id compare="ge" version="11">com.redhat.fwupdate</id>
</requires>
Also, rather than checking each requirement we know about on the component,
check each requirement on the component about things we know. This ensures we
don't allow firmware to be installs that requires for instance fwupdate 22 when
the runtime version is only being added in fwupdate 12 and up.
This means the following is now an error that will fail to allow the firmware
to be installed:
<requires>
<firmware>doesnotexist</firmware>
<some_future_tag>also_unknown</some_future_tag>
</requires>
Also add a lot of self tests to test the various new failure modes.
Fixes https://github.com/hughsie/fwupd/issues/463
There are a lot of hacks here;
* Pulling newer libappstream-glib from Fedora
* Pulling a systemd backport
* Manually installing pillow and pygobject
* PKCS7 is turned off (gnutls is too old)
Don't optimize the device flag away. We need to be able to set the flag on all
devices that match the device ID, not just the one that happens to match now.
If the device firmware was set incorrectly make then set it to the release
version so the database update works correctly. We can't do any kind of vercmp
in the database, so use a daemon warning so we can either fix the plugin or
the XML.
This fixes up the issue that the hardware reports '28.00' and the AppStream
release specifies '28.0'.
Fixes: https://github.com/hughsie/fwupd/issues/387
If this happens:
1.2.3 -> 1.2.4 = failure
1.2.3 -> 1.2.5 = success
...we want to preserve both in the history database so they can both be shared
with the LVFS. Use the device ID and the new and old firmware versions when
modifying and deleting entries.
This fix is made slightly more tricky as we have to drop the PRIMARY KEY
attribute on the device_id, and due to a limitation in SQLite, it means copying
the old history into a new table.
Previous to having the history database we could only notify about firmware that
as installed using the uefi plugin, as that had a few system-wide API calls to
say 'this update failed' or 'this was the error'.
Now we have the local history database not only can we report more details about
the UEFI update (e.g. the old version number) but we can also offer the same
functionality for all other plugins.
Although this does rework how the data for GetResults() is populated, it does
make the FuEngine object quite a lot less confused.
It also fixes a warning in the fwupd plugin for gnome-software, which was
expecting the FwupdRelease to be populated for the FwupdDevice.
This allows us to record whether we've shown the user a notification (either in
the terminal or in a GUI) that an update failed or was successful.
This can't be done in the session otherwise we'd get a notification for every
different user on the system. Notifying also isn't the same as reporting,
although one can certainly follow on from the latter.
In the case of failing to even set up UpdateCapsule, the uefi plugin would
dutifully return SUCCESS as it was referring to the 2nd-to-last update that
actually worked.
The SUPPORTED flag is used when a device appears in the AppStream metadata of
any enabled remote, so when we rescan the modified store also ensure the flag
state is still correct.
Fixes https://github.com/hughsie/fwupd/issues/363
Rename FuPending to FuHistory to better represent what the object is now doing.
Also, while we're here, switch to using SQLite prepared statements to avoid a
possible invalid read on i386 hardware.
Always set the AppStream app properties on the FwupdRelease. In some cases we
were returning FwupdRelease objects with no name or summary which gnome-software
was ignoring.
Using old versions of gcab we could only do one thing: extract the files in the
cabinet archive to a new directory in /tmp, and then fwupd would have to read
them back in to memory to parse them. This was both inelegant and wasteful, and
probably not an awesome idea from a security or privacy point of view.
Using libgcab >= 1.0 we can decompress to a GBytes blob, and then verify the
firmware and metainfo file without anything being written to disk.
As this is a security sensitive operation, move the fwupd-specific helper code
out of libappstream-glib and also add a lot of internal self tests.
The gcab code will have to remain in libappstream-glib for a long time, but we
don't have to use it. Handling the cab file here also allows us to fix two
long-standing bugs:
* MetaInfo or firmware files in a subdirectory are handled correctly
* The archive can also be self-signed using PKCS7 instead of using GPG
In the case of multiple <component> sections with different AppStream IDs, but
with the same GUID <provides>, filter using the requirements rather than just
choosing the first one.
This allows the update of Logitech devices with secure bootloaders. Many thanks
to Ogier Bouvier for identifying the problem.
This ensures we get progress events when replugging a device. Also, remove the
callbacks on the 'old' device to avoid causing multiple events on a 2nd-replug.
This makes more sense; we're updating the device, not the plugin itself.
This also means we don't need to funnel everything through callbacks like
GFileProgressCallback and we can also update the state without adding an
explicit callback to each derived device type.
`Not compatible with fwupd version 1.0.2, requires >= 1.0.3`
...is easier to understand than...
`Value of org.freedesktop.fwupd incorrect: failed predicate [1.0.3 ge 1.0.2]`
This saves all the USB plugins from connecting to the context and managing the
device lifecycle and allows devices that uses FuUsbDevice to be removed
automatically.
This makes supported plugins *much* smaller indeed.
When changing from runtime->bootloader->runtime the usual way of handling this
in a fwupd plugin is to:
* reset the device and wait for a replug
* flash the hardware
* reset the device and wait for a replug
This works well when the runtime and bootloader modes are handled by the same
plugin. For situations like the Nitrokey device, where one plugin handles the
runtime (nitrokey), and another handles the bootloader (dfu) we have to have
the ability to 'ignore' the device removal and just issue a 'changed' signal
so the client refreshes the properties.
This moves more functionality out of the engine, and will allow us to add some
cleverness to the device list to allow the FuDevice to be shared between
different plugins.
It's actually less scary to see a SHA1 hash than it is to see a path like
/sys/devices/pci0000:00/0000:00:1d.0/usb1/1-1/1-1.2. It's also way easier to
copy and paste into the various fwupdmgr command that require a device ID and
also means we can match a partial prefix much like git allows.
If we also move to a model where plugins can be changed during different stages
of the update (e.g. during detach) then the device might change connection type
and then the sysfs path not only becomes difficult to paste, but incorrect.
Session software doesn't care about the format of the device ID (it is supposed
to be an implementation detail) and so there's no API or ABI break here. A few
plugins also needed to be ported, but nothing too worrying.
This allows us to do two things:
* Attach after a failed update, so the user isn't left with 'dead' hardware
* Split the detach and attach actions into different plugins in the future
This also allows us to have a separate vfunc to get the new version number
after flashing the firmware, as this may be handled in a different plugin to
the detach phase.
In the latest version of the LVFS you can restrict the firmware to a specific
machine type, for instance a specific baseboard vendor. This is the same as
done in Microsoft Update using the CHID mechanism.
This commit adds support for the <hardware> requires type, although it needs to
be built against appstream-glib 0.7.4 to be supported and/or tested.
When fwupd is installed in long-term support distros it's very hard to backport
new versions as new hardware is released.
There are several reasons why we can't just include the mapping and quirk
information in the AppStream metadata:
* The extra data is hugely specific to the installed fwupd plugin versions
* The device-id is per-device, and the mapping is usually per-plugin
* Often the information is needed before the FuDevice is created
* There are security implications in allowing plugins to handle new devices
The idea with quirks is that the end user can drop an additional (or replace
an existing) file in a .d director with a simple format and the hardware will
magically start working. This assumes no new quirks are required, as this would
obviously need code changes, but allows us to get most existing devices working
in an easy way without the user compiling anything.
This allows us to fix issues like https://github.com/hughsie/fwupd/issues/265
This allows a plugin to see if a GUID is supported in the AppStream metadata of
configured remotes. It allows plugins to skip devices that are not supported
and that do bad things when probed.
It only remained on FwupdResult because I couldn't make up my mind about whether
it was a property of the device, or the firmware release. It's more logically
the latter, as you could have a .cab file with multiple versions of the
firmware and only the first being signed.
For a few months the lvfs-testing remote has not included firmware already
present in the stable lvfs remote. This means if you enable the testing remote
then components in the stable remote are not available to fwupd.
Instead of a simplistic check on the component ID, add the missing releases
no matter the ordering of the remotes.
This was only ever added for gnome-software, and is too inflexible for anything
else. It turns out we don't even need it in GNOME, as we can construct a
suitable ID ourselves using the existing values.
It was also ambiguous whether the unique ID was in reference to the device
or release -- and for gnome-software we need both.
The Linux DMI class still does not provide the information we need, and parsing
the blob directly also allows the Dell and Redfish plugins to get the raw data.
Over the months the original meaning of ALLOW_OFFLINE and ALLOW_ONLINE have be
lost, and there is now a confusing mixture of uses in the source tree. With this
commit we make it clear the UPDATABLE flag is used to specify when the device is
updatable (e.g. from the desktop live session, or from the systemd offline
updates mode, or both) and the NEEDS_REBOOT flag lets us know when the update
is actually going to be done.
For instance, a UEFI UpdateCapsule can be *scheduled* from either the desktop
or from the update mode (but the latter would be a bit weird), but does require
a reboot. Some devices might only be updatable outside the live session, for
instance a hard drive update or a GPU update -- there's just too much going on
with a live session and we want to tightly control what's running during the
firmware flash.
This also means we don't have to "retry" the update when scheduling an update
that really can be scheduled whenever, but just requires a reboot to apply.
This could be used, for instance, to set a property on ThunderBolt controllers
inside Dell computers saying that they support forcing the power level during
coldplug. It could also be used to set the dock type for the synapticsmst hub.
Adding this level of complexity allows us to avoid the creep of HAVE_DELL and
HAVE_LENOVO into seemingly unrelated plugins, and also allows us to have
multiple vendor plugins providing the same end result with two different
vendor-specific mechanisms.
This means we return an error when encountering a rollback attack. This can
currently be performed by providing the old metadata and old signature when
calling into UpdateMetadata.
We can use this as an alternative for GPG. No PKCS7 certificates are currently
installed by fwupd and it's expected that the LVFS will still only provide GPG
detached signatures.
If an OEM distributor wants to sign firmware with a PKCS7 and the corresponding
certificate is provided then the firmware will be marked as valid.
Only firmware shipping with a .p7b file will use the PKCS7 functionality,
similarly remote metadata validation will default to GPG unless Keyring=pkcs7
is specified in the config file.
This avoids open()ing and close()ing multiple times on hotplug -- which in
itself isn't a huge problem as the requests are refcounted in libusb, but it
matters hugely when a plugin accidentally closes a device that was not opened.
As all the devices are going to be opened anyway (to read the vendor strings)
and the cost of keeping the device is open is tiny, just get libgusb to
auto-open *all* devices and keep them open for the duration.
Fixes: https://github.com/hughsie/fwupd/issues/155
When reading with g_dir_read_name() the returned files do not have to be sorted
in any particular order and could even change between invokation. This patch
makes debugging the interactions between plugins much easier.
The items that 0.6.13 requires are now guarded by a version test.
This should allow running fwupd master on more distros that haven't
yet picked up appstream-glib 0.6.13.
We used to do this dance to avoid reading the Option ROM on hardware by default
(some faulty hardware would crash...) but now we're doing the verify update in
the daemon there's no need to split this into two steps.
Fixes: https://github.com/hughsie/fwupd/issues/149
The idea here is that we move a lot of the 'meat' out of fu-main into the
engine. This also lets us simplify a lot of things and ensures the user
authentication is simple and easy to audit.
