SWU Delta Update with AWS S3
SWUpdate supports the delta update, a feature that allows downloading only the differences rather than the entire image. This is particularly useful for small updates on large images, especially with limited or unstable connectivity. However, it is important to note that the overall update time may not necessarily be reduced, as the target device requires additional computation to determine which parts need to be downloaded
SWUpdate uses the zchunk compression format.
At build time, the rootfs is compressed (.zck) and its chunk header is calculated (.zck.zckheader).
The compressed rootfs must be hosted on an HTTP server that supports the range requests.
The chunk header is included in the swu bundle.
For a detailed explanation of the process, refer to the article Delta OTA Update with SWUpdate
This post describes how to configure SWUpdate (v2024.12) to use AWS S3 as the HTTP server for delta updates.
AWS S3 as the HTTP server
Two problems arise when using the AWS S3 as the HTTP server:
-
AWS S3 uses presigned URL, a temporary generated URL that cannot be predefined. Thus, no URL can be hardcoded in the swu-description.
-
AWS S3 HTTP server does not support the mulitple range request. Instead of returning the expected
206status code, it returns a200code, causing the update process to fail on the SWUpdate side.
To address these issues, changes to the SWUpdate source code are necessary.
Dynamic URL
The first change adds dynamic URL support. SWUpdate reads the URL from a file instead of reading it from the swu bundle.
The following patch implements this feature:
From 52899d5ed85fb1cd5a4136d1a19f9ee6ab1c1905 Mon Sep 17 00:00:00 2001
From: Pierre-Loup GOSSE <pierre-loup.gosse@smile.fr>
Date: Tue, 22 Apr 2025 14:50:11 +0200
Subject: [PATCH 1/2] Dynamic URL for delta handler if property is empty
If 'images.properties.url' is defined but empty for delta update, read
the URL from /tmp/swupdate_url file (no newline). This workaround is
for dynamic URL like presigned URL.
---
handlers/Kconfig | 9 +++++++++
handlers/delta_handler.c | 30 ++++++++++++++++++++++++++++++
2 files changed, 39 insertions(+)
diff --git a/handlers/Kconfig b/handlers/Kconfig
index f4608dc3..5370c428 100644
--- a/handlers/Kconfig
+++ b/handlers/Kconfig
@@ -92,6 +92,15 @@ config DELTA_HTTP_MONO_RANGE
If enabled, the delta handler trigger a download per range. This
adds an acceptable overhead.
+config DELTA_DYNAMIC_URL
+ bool "dynamic URL for delta update"
+ depends on DELTA
+ default n
+ help
+ If images.properties.url from swu is an empty string, replace it
+ with URL read from /tmp/swupdate_url file (no newline). This is
+ useful when using presigned URL which are dynamically generated.
+
config DISKPART
bool "diskpart"
depends on HAVE_LIBFDISK
diff --git a/handlers/delta_handler.c b/handlers/delta_handler.c
index 08e74e1e..36c2f86f 100644
--- a/handlers/delta_handler.c
+++ b/handlers/delta_handler.c
@@ -297,6 +297,27 @@ static size_t get_total_size(zckCtx *zck, struct hnd_priv *priv) {
return pos;
}
+#ifdef CONFIG_DELTA_DYNAMIC_URL
+/*
+ * Get dynamic URL from file. Useful for pre-signed URL for example.
+ */
+static char* get_dynamic_url(void) {
+ FILE *file = fopen("/tmp/swupdate_url", "r");
+ size_t len = 0;
+ ssize_t read;
+ char *line;
+
+ if (!file) {
+ WARN("Dynamic URL in /tmp/swupdate_url not found");
+ return NULL;
+ }
+
+ read = getline(&line, &len, file);
+
+ return line;
+}
+#endif
+
/*
* Get attributes from sw-description
*/
@@ -306,6 +327,15 @@ static int delta_retrieve_attributes(struct img_type *img, struct hnd_priv *priv
priv->zckloglevel = ZCK_LOG_DDEBUG;
priv->url = dict_get_value(&img->properties, "url");
+
+#ifdef CONFIG_DELTA_DYNAMIC_URL
+ if (priv->url && strlen(priv->url) == 0) {
+ priv->url = get_dynamic_url();
+ if (priv->url)
+ INFO("Use dynamic URL '%s'", priv->url);
+ }
+#endif
+
priv->srcdev = dict_get_value(&img->properties, "source");
priv->chainhandler = dict_get_value(&img->properties, "chain");
if (!priv->url || !priv->srcdev ||
--
2.34.1
If CONFIG_DELTA_DYNAMIC_URL is enabled, the delta handler will read the URL from the /tmp/swudpate_url file only if the url property is empty.
This is a proof-of-concept solution. No signature and verification is performed on the /tmp/swupdate_url file at the SWUpdate level.
Mono HTTP range request
The second change enables mono HTTP range requests. Instead of sending a single HTTP request for all missing ranges, SWUpdate sends one request per missing range. While this introductes some overhead, it is acceptable given that only a few ranges are typically missing during an update.
From 29f8faa073d300a0a27f79282952d561ecb98c14 Mon Sep 17 00:00:00 2001
From: Pierre-Loup GOSSE <pierre-loup.gosse@smile.fr>
Date: Tue, 22 Apr 2025 15:59:04 +0200
Subject: [PATCH 2/2] Support mono HTTP range request for delta update
Some HTTP server does not support multiple ranges per GET request, for
example AWS S3 server. The update failed because server response code is
not 206.
Add the DELTA_HTTP_MONO_RANGE config. If enabled, the delta handler
trigger a download per range. This adds an acceptable overhead.
---
handlers/Kconfig | 12 ++++++++++++
handlers/delta_handler.c | 7 ++++++-
2 files changed, 18 insertions(+), 1 deletion(-)
diff --git a/handlers/Kconfig b/handlers/Kconfig
index 152bc080..f4608dc3 100644
--- a/handlers/Kconfig
+++ b/handlers/Kconfig
@@ -80,6 +80,18 @@ config DELTA
and download the missing parts, and pass the resulting image to the
next handler.
+config DELTA_HTTP_MONO_RANGE
+ bool "download an HTTP range at a time"
+ depends on DELTA
+ select CHANNEL_CURL
+ default n
+ help
+ Some HTTP server does not support multiple ranges per GET request,
+ for example AWS S3 server.
+
+ If enabled, the delta handler trigger a download per range. This
+ adds an acceptable overhead.
+
config DISKPART
bool "diskpart"
depends on HAVE_LIBFDISK
diff --git a/handlers/delta_handler.c b/handlers/delta_handler.c
index c7a57a3c..08e74e1e 100644
--- a/handlers/delta_handler.c
+++ b/handlers/delta_handler.c
@@ -534,13 +534,18 @@ static bool trigger_download(struct hnd_priv *priv)
zckCtx *tgt = priv->tgt;
size_t reqlen;
zck_range *range;
+ int max_ranges = priv->max_ranges;
char *http_range;
bool status = true;
priv->boundary[0] = '\0';
- range = zchunk_get_missing_range(tgt, priv->chunk, priv->max_ranges);
+#ifdef CONFIG_DELTA_HTTP_MONO_RANGE
+ max_ranges = 1;
+#endif
+
+ range = zchunk_get_missing_range(tgt, priv->chunk, max_ranges);
if (!range)
return false;
http_range = zchunk_get_range_char(range);
--
2.34.1
To enable this feature, set CONFIG_DELTA_HTTP_MONO_RANGE to y.