Add GRUB bootloader patches for talosctl upgrade on RPi5/CM5
All checks were successful
Build Talos CM5 Image / build (push) Successful in 1h4m48s
All checks were successful
Build Talos CM5 Image / build (push) Successful in 1h4m48s
Force GRUB instead of sd-boot on arm64 and pass --no-nvram to grub-install, working around the SetVariableRT firmware limitation that prevents in-place upgrades on RPi5/CM5 hardware. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
parent
b1eb322d7b
commit
689b9402a8
36
README.md
36
README.md
@ -46,37 +46,27 @@ zstd -d metal-arm64.raw.zst -o metal-arm64.raw
|
|||||||
|
|
||||||
### Upgrade an existing node
|
### Upgrade an existing node
|
||||||
|
|
||||||
> **Warning:** In-place upgrades via `talosctl upgrade` may fail on RPi5/CM5 hardware with a `SetVariableRT` EFI firmware error. See [Known issues](#known-issues) below. For now, the recommended upgrade path is to re-flash the disk image.
|
```bash
|
||||||
|
talosctl upgrade --image docker.io/svrnty/talos-rpi5:v1.12.3-k6.12.47-2
|
||||||
|
```
|
||||||
|
|
||||||
|
> **Note:** In-place upgrades use GRUB with `--no-nvram` to work around the RPi5/CM5 `SetVariableRT` firmware limitation. This patch is included but not yet tested in production — re-flashing the disk image is the proven fallback.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Re-flash method (reliable)
|
# Fallback: re-flash method
|
||||||
zstd -d metal-arm64.raw.zst -o metal-arm64.raw
|
zstd -d metal-arm64.raw.zst -o metal-arm64.raw
|
||||||
# Flash to eMMC/SD via your preferred tool
|
# Flash to eMMC/SD via your preferred tool
|
||||||
|
|
||||||
# In-place method (experimental — may fail, see known issues)
|
|
||||||
talosctl upgrade --image docker.io/svrnty/talos-rpi5:v1.12.3-k6.12.47-2
|
|
||||||
```
|
```
|
||||||
|
|
||||||
### What's included
|
### What's included
|
||||||
|
|
||||||
- RPi downstream kernel with CM5/RP1 support (4K page size, aligned with upstream Talos)
|
- RPi downstream kernel with CM5/RP1 support (4K page size, aligned with upstream Talos)
|
||||||
|
- GRUB bootloader with `--no-nvram` for reliable `talosctl upgrade` on RPi5/CM5
|
||||||
- Overclock: 2.6GHz (`arm_freq=2600`, `over_voltage_delta=50000`, `arm_boost=1`)
|
- Overclock: 2.6GHz (`arm_freq=2600`, `over_voltage_delta=50000`, `arm_boost=1`)
|
||||||
- Extensions: `iscsi-tools`, `util-linux-tools`
|
- Extensions: `iscsi-tools`, `util-linux-tools`
|
||||||
|
|
||||||
## Known issues
|
## Known issues
|
||||||
|
|
||||||
### In-place upgrade fails (SetVariableRT)
|
|
||||||
|
|
||||||
`talosctl upgrade` may fail during the bootloader installation step with:
|
|
||||||
|
|
||||||
```
|
|
||||||
Firmware does not support SetVariableRT. Can not remount with rw
|
|
||||||
```
|
|
||||||
|
|
||||||
The RPi5/CM5 EFI firmware does not support runtime EFI variable writes, which the Talos bootloader update requires. **Re-flashing the disk image is the reliable upgrade path for now.** We are investigating GRUB-based boot as a fix (see [Roadmap](#roadmap)).
|
|
||||||
|
|
||||||
*Upstream: <a href="https://github.com/talos-rpi5/talos-builder/issues/21" target="_blank">talos-builder#21</a>*
|
|
||||||
|
|
||||||
### No serial console output after boot
|
### No serial console output after boot
|
||||||
|
|
||||||
Serial output goes silent after the EFI stub decompresses the kernel and exits boot services. This affects headless debugging on CM5 boards where serial is the primary console.
|
Serial output goes silent after the EFI stub decompresses the kernel and exits boot services. This affects headless debugging on CM5 boards where serial is the primary console.
|
||||||
@ -91,12 +81,14 @@ Talos ignores the `machine.install.disk` config field on SBC platforms. You **mu
|
|||||||
|
|
||||||
## Roadmap
|
## Roadmap
|
||||||
|
|
||||||
This project targets production-ready Talos clusters on RPi5/CM5 hardware. Key milestones:
|
This project targets production-ready Talos clusters on RPi5/CM5 hardware.
|
||||||
|
|
||||||
- [x] **Switch to 4K page size** — Aligned with upstream Talos kernel config. Reduces memory overhead and improves workload compatibility (Longhorn, jemalloc, F2FS, etc.).
|
| Status | Milestone | Description |
|
||||||
- [ ] **Reliable in-place upgrades** — Investigate GRUB-based boot or alternative bootloader strategies to work around the `SetVariableRT` firmware limitation, enabling `talosctl upgrade` on RPi5/CM5.
|
|--------|-----------|-------------|
|
||||||
- [ ] **Serial console fix** — Debug U-Boot/kernel handoff to restore serial output after EFI stub exit.
|
| Untested | **4K page size** | Aligned with upstream Talos kernel config. Reduces memory overhead and improves workload compatibility (Longhorn, jemalloc, F2FS, etc.). |
|
||||||
- [ ] **NVMe boot support** — Produce images that target NVMe directly, or document a supported NVMe boot flow.
|
| Untested | **Reliable in-place upgrades** | Force GRUB bootloader with `--no-nvram` on arm64 to work around the `SetVariableRT` firmware limitation (<a href="https://github.com/talos-rpi5/talos-builder/issues/21" target="_blank">talos-builder#21</a>). |
|
||||||
|
| Pending | **Serial console fix** | Debug U-Boot/kernel handoff to restore serial output after EFI stub exit. |
|
||||||
|
| Pending | **NVMe boot support** | Produce images that target NVMe directly, or document a supported NVMe boot flow. |
|
||||||
|
|
||||||
## Building
|
## Building
|
||||||
|
|
||||||
|
|||||||
@ -79,6 +79,8 @@ patches/
|
|||||||
siderolabs/
|
siderolabs/
|
||||||
pkgs/0001-*.patch # RPi kernel patch
|
pkgs/0001-*.patch # RPi kernel patch
|
||||||
talos/0001-*.patch # Module list patch
|
talos/0001-*.patch # Module list patch
|
||||||
|
talos/0002-*.patch # Skip NVRAM writes for GRUB on arm64
|
||||||
|
talos/0003-*.patch # Force GRUB bootloader on arm64
|
||||||
talos-rpi5/
|
talos-rpi5/
|
||||||
sbc-raspberrypi5/ # Overlay patches (Go toolchain bump)
|
sbc-raspberrypi5/ # Overlay patches (Go toolchain bump)
|
||||||
cosign.pub # Public key for verifying image attestations
|
cosign.pub # Public key for verifying image attestations
|
||||||
|
|||||||
@ -0,0 +1,33 @@
|
|||||||
|
From 2db8797af370535aba7c5694cd291bba8e6c5a67 Mon Sep 17 00:00:00 2001
|
||||||
|
From: Mathias Beaulieu-Duncan <mathias@svrnty.io>
|
||||||
|
Date: Fri, 13 Feb 2026 19:08:41 -0500
|
||||||
|
Subject: [PATCH 2/3] Skip NVRAM writes for GRUB on arm64
|
||||||
|
|
||||||
|
On arm64 platforms like RPi5/CM5, the UEFI firmware (U-Boot) does not
|
||||||
|
support EFI runtime SetVariable, causing grub-install to fail when
|
||||||
|
efibootmgr tries to create NVRAM boot entries. Since arm64-efi systems
|
||||||
|
boot by convention (finding BOOTAA64.efi on the ESP), NVRAM boot
|
||||||
|
entries are unnecessary.
|
||||||
|
|
||||||
|
Pass --no-nvram to grub-install on arm64 to allow in-place upgrades
|
||||||
|
via talosctl upgrade to succeed.
|
||||||
|
---
|
||||||
|
.../machined/pkg/runtime/v1alpha1/bootloader/grub/install.go | 2 +-
|
||||||
|
1 file changed, 1 insertion(+), 1 deletion(-)
|
||||||
|
|
||||||
|
diff --git a/internal/app/machined/pkg/runtime/v1alpha1/bootloader/grub/install.go b/internal/app/machined/pkg/runtime/v1alpha1/bootloader/grub/install.go
|
||||||
|
index 6f5c9f8..766374b 100644
|
||||||
|
--- a/internal/app/machined/pkg/runtime/v1alpha1/bootloader/grub/install.go
|
||||||
|
+++ b/internal/app/machined/pkg/runtime/v1alpha1/bootloader/grub/install.go
|
||||||
|
@@ -196,7 +196,7 @@ func (c *Config) install(opts options.InstallOptions) (*options.InstallResult, e
|
||||||
|
args = append(args, "--efi-directory="+filepath.Join(opts.MountPrefix, constants.EFIMountPoint))
|
||||||
|
}
|
||||||
|
|
||||||
|
- if opts.ImageMode {
|
||||||
|
+ if opts.ImageMode || opts.Arch == arm64 {
|
||||||
|
args = append(args, "--no-nvram")
|
||||||
|
}
|
||||||
|
|
||||||
|
--
|
||||||
|
2.50.1 (Apple Git-155)
|
||||||
|
|
||||||
@ -0,0 +1,47 @@
|
|||||||
|
From 1393b3f013e758f6bb52d14006d3a7e7db348930 Mon Sep 17 00:00:00 2001
|
||||||
|
From: Mathias Beaulieu-Duncan <mathias@svrnty.io>
|
||||||
|
Date: Fri, 13 Feb 2026 19:08:58 -0500
|
||||||
|
Subject: [PATCH 3/3] Force GRUB bootloader on arm64
|
||||||
|
|
||||||
|
On arm64 platforms like RPi5/CM5, the UEFI firmware (U-Boot) exposes
|
||||||
|
/sys/firmware/efi but does not support EFI runtime SetVariable. This
|
||||||
|
causes NewAuto() to select sd-boot, which then fails when trying to
|
||||||
|
write EFI variables during installation/upgrade.
|
||||||
|
|
||||||
|
Force GRUB on arm64 since it uses config files instead of EFI
|
||||||
|
variables for boot configuration. Combined with the --no-nvram patch,
|
||||||
|
this enables reliable in-place upgrades via talosctl upgrade on
|
||||||
|
RPi5/CM5 hardware.
|
||||||
|
|
||||||
|
Ref: https://github.com/siderolabs/talos/issues/10859
|
||||||
|
Ref: https://github.com/talos-rpi5/talos-builder/issues/21
|
||||||
|
---
|
||||||
|
.../machined/pkg/runtime/v1alpha1/bootloader/bootloader.go | 5 +++++
|
||||||
|
1 file changed, 5 insertions(+)
|
||||||
|
|
||||||
|
diff --git a/internal/app/machined/pkg/runtime/v1alpha1/bootloader/bootloader.go b/internal/app/machined/pkg/runtime/v1alpha1/bootloader/bootloader.go
|
||||||
|
index f084e09..5c388c1 100644
|
||||||
|
--- a/internal/app/machined/pkg/runtime/v1alpha1/bootloader/bootloader.go
|
||||||
|
+++ b/internal/app/machined/pkg/runtime/v1alpha1/bootloader/bootloader.go
|
||||||
|
@@ -8,6 +8,7 @@ package bootloader
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
+ goruntime "runtime"
|
||||||
|
|
||||||
|
"github.com/siderolabs/go-blockdevice/v2/block"
|
||||||
|
"github.com/siderolabs/go-blockdevice/v2/partitioning/gpt"
|
||||||
|
@@ -73,6 +74,10 @@ func Probe(disk string, options options.ProbeOptions) (Bootloader, error) {
|
||||||
|
|
||||||
|
// NewAuto returns a new bootloader based on auto-detection.
|
||||||
|
func NewAuto() Bootloader {
|
||||||
|
+ if goruntime.GOARCH == "arm64" {
|
||||||
|
+ return grub.NewConfig()
|
||||||
|
+ }
|
||||||
|
+
|
||||||
|
if sdboot.IsUEFIBoot() {
|
||||||
|
return sdboot.New()
|
||||||
|
}
|
||||||
|
--
|
||||||
|
2.50.1 (Apple Git-155)
|
||||||
|
|
||||||
Loading…
Reference in New Issue
Block a user