Some UEFI firmware is easily provoked into running out of space in its
variable storage. This is usually due to certain kernel drivers (e.g.
pstore), but regardless of the cause it can cause grub-install to fail
because it currently asks efibootmgr to delete and re-add entries, and
the deletion often doesn't result in an immediate garbage collection.
Writing variables frequently also increases wear on the NVRAM which may
have limited write cycles. For these reasons, it's desirable to find a
way to minimise writes while still allowing grub-install to ensure that
a suitable boot entry exists.
Unfortunately, efibootmgr doesn't offer an interface that would let
grub-install do this. It doesn't in general make very much effort to
minimise writes; it doesn't allow modifying an existing Boot* variable
entry, except in certain limited ways; and current versions don't have a
way to export the expected variable data so that grub-install can
compare it to the current data. While it would be possible (and perhaps
desirable?) to add at least some of this to efibootmgr, that would still
leave the problem that there isn't a good upstreamable way for
grub-install to guarantee that it has a new enough version of
efibootmgr. In any case, it's cumbersome and slow for grub-install to
have to fork efibootmgr to get things done.
Fortunately, a few years ago Peter Jones helpfully factored out a
substantial part of efibootmgr to the efivar and efiboot libraries, and
so it's now possible to have grub-install use those directly. We still
have to use some code from efibootmgr, but much less than would
previously have been necessary.
grub-install now reuses existing boot entries where possible, and avoids
writing to variables when the new contents are the same as the old
contents. In the common upgrade case where nothing needs to change, it
no longer writes to NVRAM at all. It's also now slightly faster, since
using libefivar is faster than forking efibootmgr.
Fixes Debian bug #891434.
Signed-off-by: Colin Watson <cjwatson@ubuntu.com>
Bug-Debian: https://bugs.debian.org/891434
Forwarded: https://lists.gnu.org/archive/html/grub-devel/2019-03/msg00119.html
Last-Update: 2019-03-23
Patch-Name: efi-variable-storage-minimise-writes.patch
Some UEFI firmware is easily provoked into running out of space in its
variable storage. This is usually due to certain kernel drivers (e.g.
pstore), but regardless of the cause it can cause grub-install to fail
because it currently asks efibootmgr to delete and re-add entries, and
the deletion often doesn't result in an immediate garbage collection.
Writing variables frequently also increases wear on the NVRAM which may
have limited write cycles. For these reasons, it's desirable to find a
way to minimise writes while still allowing grub-install to ensure that
a suitable boot entry exists.
Unfortunately, efibootmgr doesn't offer an interface that would let
grub-install do this. It doesn't in general make very much effort to
minimise writes; it doesn't allow modifying an existing Boot* variable
entry, except in certain limited ways; and current versions don't have a
way to export the expected variable data so that grub-install can
compare it to the current data. While it would be possible (and perhaps
desirable?) to add at least some of this to efibootmgr, that would still
leave the problem that there isn't a good upstreamable way for
grub-install to guarantee that it has a new enough version of
efibootmgr. In any case, it's cumbersome and slow for grub-install to
have to fork efibootmgr to get things done.
Fortunately, a few years ago Peter Jones helpfully factored out a
substantial part of efibootmgr to the efivar and efiboot libraries, and
so it's now possible to have grub-install use those directly. We still
have to use some code from efibootmgr, but much less than would
previously have been necessary.
grub-install now reuses existing boot entries where possible, and avoids
writing to variables when the new contents are the same as the old
contents. In the common upgrade case where nothing needs to change, it
no longer writes to NVRAM at all. It's also now slightly faster, since
using libefivar is faster than forking efibootmgr.
Fixes Debian bug #891434.
Signed-off-by: Colin Watson <cjwatson@ubuntu.com>
Bug-Debian: https://bugs.debian.org/891434
Forwarded: https://lists.gnu.org/archive/html/grub-devel/2019-03/msg00119.html
Last-Update: 2019-03-23
Patch-Name: efi-variable-storage-minimise-writes.patch
This patch is similiar to commit e795b9011 (RISC-V: Add libgcc helpers
for clz) but for SPARC target.
Signed-off-by: Daniel Kiper <daniel.kiper@oracle.com>
Reviewed-by: Ross Philipson <ross.philipson@oracle.com>
This patch is similiar to commit e795b9011 (RISC-V: Add libgcc helpers
for clz) but for MIPS target.
Signed-off-by: Daniel Kiper <daniel.kiper@oracle.com>
Reviewed-by: Ross Philipson <ross.philipson@oracle.com>
MIPS fallout cleanup after commit 4d4a8c96e (verifiers: Add possibility
to verify kernel and modules command lines).
Signed-off-by: Daniel Kiper <daniel.kiper@oracle.com>
Reviewed-by: Ross Philipson <ross.philipson@oracle.com>
PowerPC fallout cleanup after commit 4d4a8c96e (verifiers: Add possibility
to verify kernel and modules command lines) and ca0a4f689 (verifiers: File
type for fine-grained signature-verification controlling).
Signed-off-by: Daniel Kiper <daniel.kiper@oracle.com>
Reviewed-by: Ross Philipson <ross.philipson@oracle.com>
IA-64 fallout cleanup after commit 4d4a8c96e (verifiers: Add possibility
to verify kernel and modules command lines).
Signed-off-by: Daniel Kiper <daniel.kiper@oracle.com>
Reviewed-by: Ross Philipson <ross.philipson@oracle.com>
In addition to what was already there, Gnulib's <intprops.h> needs SCHAR_MIN,
SCHAR_MAX, SHRT_MIN, INT_MIN, LONG_MIN, and LONG_MAX. Fixes build on CentOS 7.
Reported-by: "Chen, Farrah" <farrah.chen@intel.com>
Signed-off-by: Colin Watson <cjwatson@ubuntu.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
The sparse inode metadata format became a mkfs.xfs default in
xfsprogs-4.16.0, and such filesystems are now rejected by grub as
containing an incompatible feature.
In essence, this feature allows xfs to allocate inodes into fragmented
freespace. (Without this feature, if xfs could not allocate contiguous
space for 64 new inodes, inode creation would fail.)
In practice, the disk format change is restricted to the inode btree,
which as far as I can tell is not used by grub. If all you're doing
today is parsing a directory, reading an inode number, and converting
that inode number to a disk location, then ignoring this feature
should be fine, so I've added it to XFS_SB_FEAT_INCOMPAT_SUPPORTED
I did some brief testing of this patch by hacking up the regression
tests to completely fragment freespace on the test xfs filesystem, and
then write a large-ish number of inodes to consume any existing
contiguous 64-inode chunk. This way any files the grub tests add and
traverse would be in such a fragmented inode allocation. Tests passed,
but I'm not sure how to cleanly integrate that into the test harness.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Tested-by: Chris Murphy <lists@colorremedies.com>
Origin: upstream, https://git.savannah.gnu.org/cgit/grub.git/commit/?id=cda0a857dd7a27cd5d621747464bfe71e8727fff
Bug-Debian: https://bugs.debian.org/924760
Last-Update: 2019-03-20
Patch-Name: xfs-sparse-inodes.patch
Mirror behaviour of ELF loader in libxc: first look for Xen notes in
PT_NOTE segment, then in SHT_NOTE section and only then fallback to
a section with __xen_guest name. This fixes loading PV kernels that
Xen note have outside of PT_NOTE. While this may be result of a buggy
linker script, loading such kernel directly works fine, so make it work
with GRUB too. Specifically, this applies to binaries built from Unikraft.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Various GRUB utilities fail if the current directory doesn't exist,
because grub_find_device() chdirs to a different directory and then
fails when trying to chdir back. Gnulib's save-cwd module uses fchdir()
instead when it can, avoiding this category of problem.
Fixes Debian bug #918700.
Signed-off-by: Colin Watson <cjwatson@ubuntu.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Mostly for cosmetic reasons, we add a "net_dhcp" command, which is (at the
moment) identical to the existing "net_bootp" command. Both actually trigger
a DHCP handshake now, and both should be able to deal with pure BOOTP servers.
We could think about dropping the DHCP options from the initial DISCOVER packet
when the user issues the net_bootp command, but it's unclear whether this is
really useful, as both protocols should be able to coexist.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Even though we were parsing some DHCP options sent by the server, so far
we are only using the BOOTP 2-way handshake, even when talking to a DHCP
server.
Change this by actually sending out DHCP DISCOVER packets instead of the
generic (mostly empty) BOOTP BOOTREQUEST packets.
A pure BOOTP server would ignore the extra DHCP options in the DISCOVER
packet and would just reply with a BOOTREPLY packet, which we also
handle in the code.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
In respone to a BOOTREQUEST packet a BOOTP server would answer with a BOOTREPLY
packet, which ends the conversation for good. DHCP uses a 4-way handshake,
where the initial server respone is an OFFER, which has to be answered with
REQUEST by the client again, only to be completed by an ACKNOWLEDGE packet
from the server.
Teach the grub_net_process_dhcp() function to deal with OFFER packets,
and treat ACK packets the same es BOOTREPLY packets.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
The BOOTP RFC describes the boot file name and the server name as being part
of the integral BOOTP data structure, with some limits on the size of them.
DHCP extends this by allowing them to be separate DHCP options, which is more
flexible.
Teach the code dealing with those fields to check for those DHCP options first
and use this information, if provided. We fall back to using the BOOTP
information if those options are not used.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Currently we have a global timeout for all network cards in the BOOTP/DHCP
discovery process.
Make this timeout a per-interface one, so better accommodate the upcoming
4-way DHCP handshake and to also cover the lease time limit a DHCP offer
will come with.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Change the interface of the function dealing with incoming BOOTP packets
to take an interface instead of a card, to allow more fine per-interface
state (timeout, handshake state) later on.
Use the opportunity to clean up the code a bit.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
In contrast to BOOTP, DHCP uses a 4-way handshake, so requires to send
packets more often.
Refactor the generation and sending of the BOOTREQUEST packet into
a separate function, so that future code can more easily reuse this.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
DHCP specifies a special dummy option OVERLOAD, to allow DHCP options to
spill over into the (legacy) BOOTFILE and SNAME fields.
Parse and handle this option properly.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
For proper DHCP support we will need to parse DHCP options from a packet
more often and at various places.
Refactor the option parsing into a new function, which will scan a packet to
find *a particular* option field. Use that new function in places where we
were dealing with DHCP options before.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
The comment is right, the "giaddr" fields holds the IP address of the BOOTP
relay, not a general purpose router address. Just remove the commented code,
archeologists can find it in the git history.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
In order to be able to read from and write to model-specific registers,
two new modules are added. They are i386 specific, as the cpuid module.
rdmsr module registers the command rdmsr that allows reading from a MSR.
wrmsr module registers the command wrmsr that allows writing to a MSR.
wrmsr module is disabled if UEFI secure boot is enabled.
Please note that on SMP systems, interacting with a MSR that has a scope
per hardware thread, implies that the value only applies to the
particular cpu/core/thread that ran the command.
Also, if you specify a reserved or unimplemented MSR address, it will
cause a general protection exception (which is not currently being
handled) and the system will reboot.
Signed-off-by: Jesús Diéguez Fernández <jesusdf@gmail.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
In order to maintain the coding style consistency, it was requested to
replace the methods that use "__asm__ __volatile__" with "asm volatile".
Signed-off-by: Jesús Diéguez Fernández <jesusdf@gmail.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Add BIOS Boot Partition support for sparc64 platforms. This will work a
little different than x86. With GPT, both the OBP "load" and "boot" commands
are partition aware and neither command can see the partition table. Therefore
the entire boot-loader is stored within the BIOS Boot Partition and nothing
is stored within the bootstrap code area of MBR.
To use it, the end user will issue the boot command with the path pointing to
the BIOS Boot Partition.
For example with the disk below:
Model: Unknown (unknown)
Disk /dev/nvme1n1: 1600GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Number Start End Size File system Name Flags
1 1049kB 1075MB 1074MB ext3
2 1075MB 1076MB 1049kB bios_grub
3 1076MB 1600GB 1599GB lvm
To boot grub2 from OBP, you would use:
boot /pci@302/pci@1/pci@0/pci@13/nvme@0/disk@1:b
Signed-off-by: Eric Snowberg <eric.snowberg@oracle.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Add a new disk driver called obdisk for IEEE1275 platforms. Currently
the only platform using this disk driver is SPARC, however other IEEE1275
platforms could start using it if they so choose. While the functionality
within the current IEEE1275 ofdisk driver may be suitable for PPC and x86, it
presented too many problems on SPARC hardware.
Within the old ofdisk, there is not a way to determine the true canonical
name for the disk. Within Open Boot, the same disk can have multiple names
but all reference the same disk. For example the same disk can be referenced
by its SAS WWN, using this form:
/pci@302/pci@2/pci@0/pci@17/LSI,sas@0/disk@w5000cca02f037d6d,0
It can also be referenced by its PHY identifier using this form:
/pci@302/pci@2/pci@0/pci@17/LSI,sas@0/disk@p0
It can also be referenced by its Target identifier using this form:
/pci@302/pci@2/pci@0/pci@17/LSI,sas@0/disk@0
Also, when the LUN=0, it is legal to omit the ,0 from the device name. So with
the disk above, before taking into account the device aliases, there are 6 ways
to reference the same disk.
Then it is possible to have 0 .. n device aliases all representing the same disk.
Within this new driver the true canonical name is determined using the the
IEEE1275 encode-unit and decode-unit commands when address_cells == 4. This
will determine the true single canonical name for the device so multiple ihandles
are not opened for the same device. This is what frequently happens with the old
ofdisk driver. With some devices when they are opened multiple times it causes
the entire system to hang.
Another problem solved with this driver is devices that do not have a device
alias can be booted and used within GRUB. Within the old ofdisk, this was not
possible, unless it was the original boot device. All devices behind a SAS
or SCSI parent can be found. Within the old ofdisk, finding these disks
relied on there being an alias defined. The alias requirement is not
necessary with this new driver. It can also find devices behind a parent
after they have been hot-plugged. This is something that is not possible
with the old ofdisk driver.
The old ofdisk driver also incorrectly assumes that the device pointing to by a
device alias is in its true canonical form. This assumption is never made with
this new driver.
Another issue solved with this driver is that it properly caches the ihandle
for all open devices. The old ofdisk tries to do this by caching the last
opened ihandle. However this does not work properly because the layer above
does not use a consistent device name for the same disk when calling into the
driver. This is because the upper layer uses the bootpath value returned within
/chosen, other times it uses the device alias, and other times it uses the
value within grub.cfg. It does not have a way to figure out that these devices
are the same disk. This is not a problem with this new driver.
Due to the way GRUB repeatedly opens and closes the same disk. Caching the
ihandle is important on SPARC. Without caching, some SAS devices can take
15 - 20 minutes to get to the GRUB menu. This ihandle caching is not possible
without correctly having the canonical disk name.
When available, this driver also tries to use the deblocker #blocks and
a way of determining the disk size.
Finally and probably most importantly, this new driver is also capable of
seeing all partitions on a GPT disk. With the old driver, the GPT
partition table can not be read and only the first partition on the disk
can be seen.
Signed-off-by: Eric Snowberg <eric.snowberg@oracle.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
By default all file system modules are added to the GRUB coreboot
payload `default_payload.elf`. This makes the image quite big,
especially as often not all modules are needed.
Introduce the variable `FS_PAYLOAD_MODULES`, which can be used to
explicitly set file systems modules to be added.
$ make default_payload.elf
test -f default_payload.elf && rm default_payload.elf || true
pkgdatadir=. ./grub-mkstandalone --grub-mkimage=./grub-mkimage -O i386-coreboot -o default_payload.elf --modules='ahci pata ehci uhci ohci usb_keyboard usbms part_msdos ext2 fat at_keyboard part_gpt usbserial_usbdebug cbfs' --install-modules='ls linux search configfile normal cbtime cbls memrw iorw minicmd lsmmap lspci halt reboot hexdump pcidump regexp setpci lsacpi chain test serial multiboot cbmemc linux16 gzio echo help syslinuxcfg xnu affs afs bfs btrfs cbfs cpio cpio_be exfat ext2 f2fs fat hfs hfsplus iso9660 jfs minix minix2 minix2_be minix3 minix3_be minix_be newc nilfs2 ntfs odc procfs reiserfs romfs sfs squash4 tar udf ufs1 ufs1_be ufs2 xfs zfs password_pbkdf2 ' --fonts= --themes= --locales= -d grub-core/ /boot/grub/grub.cfg=./coreboot.cfg
$ ls -l default_payload.elf
-rw-rw---- 1 joey joey 1199568 Mar 6 13:58 default_payload.elf
$ make default_payload.elf FS_PAYLOAD_MODULES="" # ext2 already in `--modules`
test -f default_payload.elf && rm default_payload.elf || true
pkgdatadir=. ./grub-mkstandalone --grub-mkimage=./grub-mkimage -O i386-coreboot -o default_payload.elf --modules='ahci pata ehci uhci ohci usb_keyboard usbms part_msdos ext2 fat at_keyboard part_gpt usbserial_usbdebug cbfs' --install-modules='ls linux search configfile normal cbtime cbls memrw iorw minicmd lsmmap lspci halt reboot hexdump pcidump regexp setpci lsacpi chain test serial multiboot cbmemc linux16 gzio echo help syslinuxcfg xnu password_pbkdf2 ' --fonts= --themes= --locales= -d grub-core/ /boot/grub/grub.cfg=./coreboot.cfg
$ ls -l default_payload.elf
-rw-rw---- 1 joey joey 832976 Mar 7 12:13 default_payload.elf
So, the resulting payload size is around 370 kB smaller. (Adding it to
the CBFS, it will be compressed, so the effective size difference will
be smaller.)
Signed-off-by: Paul Menzel <pmenzel@molgen.mpg.de>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Upgrade Gnulib files to 20190105.
It's much easier to maintain GRUB's use of portability support files
from Gnulib when the process is automatic and driven by a single
configuration file, rather than by maintainers occasionally running
gnulib-tool and committing the result. Removing these
automatically-copied files from revision control also removes the
temptation to hack the output in ways that are difficult for future
maintainers to follow. Gnulib includes a "bootstrap" program which is
designed for this.
The canonical way to bootstrap GRUB from revision control is now
"./bootstrap", but "./autogen.sh" is still useful if you just want to
generate the GRUB-specific parts of the build system.
GRUB now requires Autoconf >= 2.63 and Automake >= 1.11, in line with
Gnulib.
Gnulib source code is now placed in grub-core/lib/gnulib/ (which should
not be edited directly), and GRUB's patches are in
grub-core/lib/gnulib-patches/. I've added a few notes to the developer
manual on how to maintain this.
Signed-off-by: Colin Watson <cjwatson@ubuntu.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
syslinux_parse simplifies some filenames by removing things like ".."
segments, but the tests assumed that @abs_top_srcdir@ would be
untouched, which is not true in the case of out-of-tree builds where
@abs_top_srcdir@ may contain ".." segments.
Performing the substitution requires some awkwardness in Makefile.am due
to details of how config.status works.
Signed-off-by: Colin Watson <cjwatson@ubuntu.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Many of GRUB's utilities don't check anywhere near all the possible
write errors. For example, if grub-install runs out of space when
copying a file, it won't notice. There were missing checks for the
return values of write, fflush, fsync, and close (or the equivalents on
other OSes), all of which must be checked.
I tried to be consistent with the existing logging practices of the
various hostdisk implementations, but they weren't entirely consistent
to start with so I used my judgement. The result at least looks
reasonable on GNU/Linux when I provoke a write error:
Installing for x86_64-efi platform.
grub-install: error: cannot copy `/usr/lib/grub/x86_64-efi-signed/grubx64.efi.signed' to `/boot/efi/EFI/debian/grubx64.efi': No space left on device.
There are more missing checks in other utilities, but this should fix
the most critical ones.
Fixes Debian bug #922741.
Signed-off-by: Colin Watson <cjwatson@ubuntu.com>
Reviewed-by: Steve McIntyre <93sam@debian.org>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
For EBR partitions, "start" is the relative starting sector of the EBR
header itself, whereas "offset" is the relative starting byte of the
partition's contents, excluding the EBR header and any padding. Thus we
must use "offset", and divide by the sector size to convert to sectors.
Fixes Debian bug #923253.
Signed-off-by: James Clarke <jrtc27@jrtc27.com>
Reviewed-by: Colin Watson <cjwatson@ubuntu.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Many of GRUB's utilities don't check anywhere near all the possible
write errors. For example, if grub-install runs out of space when
copying a file, it won't notice. There were missing checks for the
return values of write, fflush, fsync, and close (or the equivalents on
other OSes), all of which must be checked.
I tried to be consistent with the existing logging practices of the
various hostdisk implementations, but they weren't entirely consistent
to start with so I used my judgement. The result at least looks
reasonable on GNU/Linux when I provoke a write error:
Installing for x86_64-efi platform.
grub-install: error: cannot copy `/usr/lib/grub/x86_64-efi-signed/grubx64.efi.signed' to `/boot/efi/EFI/debian/grubx64.efi': No space left on device.
There are more missing checks in other utilities, but this should fix
the most critical ones.
Bug-Debian: https://bugs.debian.org/922741
Forwarded: https://lists.gnu.org/archive/html/grub-devel/2019-02/msg00125.html
Last-Update: 2019-02-28
Patch-Name: util-check-errors.patch
For EBR partitions, "start" is the relative starting sector of the EBR
header itself, whereas "offset" is the relative starting byte of the
partition's contents, excluding the EBR header and any padding. Thus we
must use "offset", and divide by the sector size to convert to sectors.
Bug-Debian: https://bugs.debian.org/923253
Forwarded: https://lists.gnu.org/archive/html/grub-devel/2019-02/msg00124.html
Last-Update: 2019-02-26
Patch-Name: freebsd-ebr-partition-offset.patch
