diff --git a/Provider/.agents/rules/how-to-work-with-user.md b/Provider/.agents/rules/how-to-work-with-user.md new file mode 100644 index 0000000..35bdc9e --- /dev/null +++ b/Provider/.agents/rules/how-to-work-with-user.md @@ -0,0 +1,5 @@ +--- +trigger: always_on +--- + +The way you must work with the human user is simple. When you finish a task, tell him what you did, what you think you should do next and ask for review and confirmation. Never go rogue. \ No newline at end of file diff --git a/Provider/Documentation/build_bundle.md b/Provider/Documentation/build_bundle.md new file mode 100644 index 0000000..a699f89 --- /dev/null +++ b/Provider/Documentation/build_bundle.md @@ -0,0 +1,59 @@ +# Universal OTA Bundle + +The Universal OTA Bundle allows you to update both the **Firmware** and the **Frontend** of the Calendink Provider in a single operation. This ensures that your UI and backend logic are always in sync. + +## 1. How it Works + +The bundle is a custom `.bundle` file that contains: +1. A **12-byte header** (Magic `BNDL`, FW size, UI size). +2. The **Firmware binary** (`Provider.bin`). +3. The **Frontend LittleFS binary** (`www_v*.bin`). + +The ESP32 backend streams this file, writing the firmware to the next OTA slot and the frontend to the inactive `www` partition. It only commits the update if both parts are written successfully. + +## 2. Prerequisites + +- You have a working [Frontend Build Environment](build_frontend.md). +- You have the [ESP-IDF SDK](https://docs.espressif.com/projects/esp-idf/en/latest/esp32/get-started/index.html) installed for firmware compilation. + +## 3. Creating a Bundle + +To create a new bundle, follow these steps in order: + +### Step A: Build the Frontend +Inside the `frontend/` directory: +```bash +npm run build:esp32 +``` + +### Step B: Build the Firmware +From the project **root** directory: +```bash +idf.py build +``` + +### Step C: Generate the Bundle +Inside the `frontend/` directory: +```bash +npm run ota:bundle +``` +> [!NOTE] +> `npm run ota:bundle` now automatically runs `npm run ota:package` first to ensure the latest Svelte build is turned into a LittleFS image before bundling. + +The output will be saved in `frontend/bundles/` with a name like `universal_v0.1.11.bundle`. + +## 4. Flashing the Bundle + +1. Open the Calendink Provider Dashboard in your browser. +2. Navigate to the **System Updates** section. +3. Click the **Universal Bundle** button. +4. Drag and drop your `.bundle` file into the upload area. +5. Click **Update**. + +The device will reboot once the upload is complete. You can verify the update by checking the version numbers and the UI changes (like the number of rockets in the header!). + +## 5. Troubleshooting + +- **"Invalid bundle magic"**: Ensure you are uploading a `.bundle` file, not a `.bin`. +- **"Firmware part is corrupted"**: The bundle was likely created while the firmware build was incomplete or failed. +- **Old UI appearing**: Ensure you ran `npm run build:esp32` *before* `npm run ota:bundle`. diff --git a/Provider/Documentation/build_frontend.md b/Provider/Documentation/build_frontend.md index b29b728..665c450 100644 --- a/Provider/Documentation/build_frontend.md +++ b/Provider/Documentation/build_frontend.md @@ -80,3 +80,9 @@ Once the backend supports it (Phase 2+), you can update the frontend without usi - Go to the **Frontend Update** section. - Select the `www.bin` file and click **Flash**. - The device will automatically write to the inactive partition and reboot. + +## 6. Universal OTA Bundle + +For a safer and more convenient update experience, you can bundle both the Firmware and Frontend into a single file. + +See the [Universal OTA Bundle Guide](build_bundle.md) for details. diff --git a/Provider/frontend/bundles/universal_v0.1.12.bundle b/Provider/frontend/bundles/universal_v0.1.12.bundle new file mode 100644 index 0000000..6c86e7f Binary files /dev/null and b/Provider/frontend/bundles/universal_v0.1.12.bundle differ diff --git a/Provider/frontend/bundles/universal_v0.1.13.bundle b/Provider/frontend/bundles/universal_v0.1.13.bundle new file mode 100644 index 0000000..0b8e1de Binary files /dev/null and b/Provider/frontend/bundles/universal_v0.1.13.bundle differ diff --git a/Provider/frontend/bundles/universal_v0.1.9.bundle b/Provider/frontend/bundles/universal_v0.1.9.bundle new file mode 100644 index 0000000..4ee4940 Binary files /dev/null and b/Provider/frontend/bundles/universal_v0.1.9.bundle differ diff --git a/Provider/frontend/package.json b/Provider/frontend/package.json index c4265e3..ec29239 100644 --- a/Provider/frontend/package.json +++ b/Provider/frontend/package.json @@ -8,6 +8,7 @@ "build": "vite build", "build:esp32": "vite build && node scripts/gzip.js", "ota:package": "node scripts/package.js", + "ota:bundle": "npm run ota:package && node scripts/bundle.js", "preview": "vite preview" }, "devDependencies": { diff --git a/Provider/frontend/scripts/bundle.js b/Provider/frontend/scripts/bundle.js new file mode 100644 index 0000000..4ca8d20 --- /dev/null +++ b/Provider/frontend/scripts/bundle.js @@ -0,0 +1,79 @@ +/** + * Universal OTA Bundle Creator + * Packs FW (Provider.bin) and WWW (www_v*.bin) into a single .bundle file. + */ + +import { readFileSync, writeFileSync, readdirSync, existsSync, mkdirSync } from 'fs'; +import { resolve, dirname } from 'path'; +import { fileURLToPath } from 'url'; + +const __dirname = dirname(fileURLToPath(import.meta.url)); +const projectRoot = resolve(__dirname, '..'); +const providerRoot = resolve(projectRoot, '..'); +const binDir = resolve(projectRoot, 'bin'); + +// Paths +const fwFile = resolve(providerRoot, 'build', 'Provider.bin'); +const bundleDir = resolve(projectRoot, 'bundles'); + +if (!existsSync(bundleDir)) { + mkdirSync(bundleDir, { recursive: true }); +} + +console.log('--- Universal Bundle Packaging ---'); + +// 1. Find the latest www.bin with proper semantic version sorting +const binFiles = readdirSync(binDir) + .filter(f => f.startsWith('www_v') && f.endsWith('.bin')) + .sort((a, b) => { + const getParts = (s) => { + const m = s.match(/v(\d+)\.(\d+)\.(\d+)/); + return m ? m.slice(1).map(Number) : [0, 0, 0]; + }; + const [aMajor, aMinor, aRev] = getParts(a); + const [bMajor, bMinor, bRev] = getParts(b); + return (bMajor - aMajor) || (bMinor - aMinor) || (bRev - aRev); + }); + +if (binFiles.length === 0) { + console.error('Error: No www_v*.bin found in frontend/bin/. Run "npm run ota:package" first.'); + process.exit(1); +} + +const wwwFile = resolve(binDir, binFiles[0]); + +if (!existsSync(fwFile)) { + console.error(`Error: Firmware binary not found at ${fwFile}. Run "idf.py build" first.`); + process.exit(1); +} + +try { + console.log(`Packing Firmware: ${fwFile}`); + console.log(`Packing Frontend: ${wwwFile}`); + + const fwBuf = readFileSync(fwFile); + const wwwBuf = readFileSync(wwwFile); + + // Create 12-byte header + // Magic: BNDL (4 bytes) + // FW Size: uint32 (4 bytes) + // WWW Size: uint32 (4 bytes) + const header = Buffer.alloc(12); + header.write('BNDL', 0); + header.writeUInt32LE(fwBuf.length, 4); + header.writeUInt32LE(wwwBuf.length, 8); + + const bundleBuf = Buffer.concat([header, fwBuf, wwwBuf]); + const outputFile = resolve(bundleDir, `universal_v${binFiles[0].replace('www_v', '').replace('.bin', '')}.bundle`); + + writeFileSync(outputFile, bundleBuf); + + console.log('-------------------------------'); + console.log(`Success: Bundle created at ${outputFile}`); + console.log(`Total size: ${(bundleBuf.length / 1024 / 1024).toFixed(2)} MB`); + console.log('-------------------------------'); + +} catch (e) { + console.error('Error creating bundle:', e.message); + process.exit(1); +} diff --git a/Provider/frontend/src/App.svelte b/Provider/frontend/src/App.svelte index 0ebfca6..6544020 100644 --- a/Provider/frontend/src/App.svelte +++ b/Provider/frontend/src/App.svelte @@ -1,5 +1,5 @@ -
-
+
+
-
-

Calendink Provider 🚀

+
+

Calendink Provider 🚀🚀👑

ESP32-S3 System Dashboard v{__APP_VERSION__}

-
- - -
- {#if status === "loading"} -
- - Connecting to ESP32... -
- {:else if status === "ok"} -
- - Connected -
- {:else if status === "rebooting"} -
- - Rebooting... -
- {:else} -
- - Offline — {errorMsg} -
- {/if} -
- - -
-
-

- System Info -

-
-
- {#each infoItems as item} -
-
- {item.icon} - {item.label} -
- - {#if status === "loading"} - - {:else} - {item.value} - {/if} - + + +
+ {#if status === "loading"} +
+ + Connecting...
- {/each} + {:else if status === "ok"} +
+ + Connected +
+ {:else if status === "rebooting"} +
+ + Rebooting... +
+ {:else} +
+ + Offline — {errorMsg} +
+ {/if}
- -
-
-
-

- Device Control -

-

- Restart the ESP32 microcontroller -

+ +
+ + +
+ +
+
+

+ System Info +

+
+
+ {#each infoItems as item} +
+
+ {item.icon} + {item.label} +
+ + {#if status === "loading"} + + {:else} + {item.value} + {/if} + +
+ {/each} +
- -
-
- - (status = "rebooting")} /> + +
+
+

+ Partition Table +

+ Flash: 16MB +
+
+ {#if status === "loading"} +
Loading memory layout...
+ {:else} + {#each otaStatus.partitions as part} +
+
+
+ + {part.label} + + {#if part.label === otaStatus.active_partition || part.label === otaStatus.running_firmware_label} + Active + {/if} +
+ + Type {part.type} / Sub {part.subtype} + +
+
+
{formatBytes(part.size)}
+
+ {#if part.app_version} + v{part.app_version} + {/if} + {#if part.free !== undefined} + + {formatBytes(part.free)} free + + {/if} +
+
+
+ {/each} + {/if} +
+
+
+ + +
+ +
+
+
+

+ Device Control +

+

+ Restart the ESP32 microcontroller +

+
+ +
+
+ + + { status = "rebooting"; isRecovering = true; }} /> +
+ +
{#if showRebootConfirm} -
-
-

- Confirm Reboot -

+
+
+

Confirm Reboot

- Are you sure you want to reboot the ESP32? The device will be - temporarily unavailable. + Are you sure you want to reboot the ESP32? The device will be temporarily unavailable.

diff --git a/Provider/frontend/src/lib/OTAUpdate.svelte b/Provider/frontend/src/lib/OTAUpdate.svelte index 905537a..1b3aa58 100644 --- a/Provider/frontend/src/lib/OTAUpdate.svelte +++ b/Provider/frontend/src/lib/OTAUpdate.svelte @@ -1,28 +1,36 @@ + + function toggleMode(mode) { + if (showAdvanced && updateMode === mode) { + showAdvanced = false; + } else { + showAdvanced = true; + updateMode = mode; + selectedFile = null; + errorMsg = ""; + } + } + + const currentTarget = $derived(() => { + if (updateMode === 'bundle') return 'FW + UI'; + if (updateMode === 'frontend') return otaInfo.target_partition; + // For firmware, target is the slot that is NOT the running one + const runningLabel = otaInfo.running_firmware_label || 'ota_0'; + return runningLabel === 'ota_0' ? 'ota_1' : 'ota_0'; + }); + - {#if !IS_DEV} -
-
-

- Frontend Info -

- -
- -
- -
-
-
Version
-
v{__APP_VERSION__}
-
-
-
Active Slot
-
- {otaInfo.active_partition} - {#if otaInfo.partitions} - - ({(otaInfo.partitions.find(p => p.label === otaInfo.active_partition)?.free / 1024 / 1024).toFixed(2)} MB free) - - {/if} -
-
-
- - {#if showAdvanced} -
-
-

OTA Upgrade

-
- Target: {otaInfo.target_partition} - {#if otaInfo.partitions} - - ({(otaInfo.partitions.find(p => p.label === otaInfo.target_partition)?.size / 1024 / 1024).toFixed(2)} MB capacity) - - {/if} -
-
- - {#if status === "success"} -
- - Update successful! The device is rebooting... -
- {:else} - -
{ e.preventDefault(); isDragging = true; }} - ondragleave={() => isDragging = false} - ondrop={handleDrop} - > - - -
📦
- {#if selectedFile} -
{selectedFile.name}
-
{(selectedFile.size / 1024).toFixed(1)} KB
- {:else} -
Drag & Drop .bin here
-
or click to browse
- {/if} -
- - {#if selectedFile} - - {/if} - - {#if status === "uploading"} -
-
-
- {:else if status === "error"} -

- {errorMsg} -

- {/if} - {/if} -
- {/if} +{#if !IS_DEV} +
+
+

+ Updates & Maintenance +

+
+ + +
- {/if} + +
+
+
+
UI Version
+
v{__APP_VERSION__}
+
+ Slot: {otaInfo.active_partition} + {#if otaInfo.partitions} + ({(otaInfo.partitions.find(p => p.label === otaInfo.active_partition)?.free / 1024).toFixed(0)} KB free) + {/if} +
+
+
+
FW Version
+
{systemInfo.firmware}
+
+ Active: {otaInfo.active_slot === 0 ? 'ota_0' : 'ota_1'} +
+
+
+ + {#if showAdvanced} +
+
+

+ OTA Upgrade ({updateMode === 'frontend' ? 'UI' : 'Firmware'}) +

+
+ Target: {currentTarget()} + {#if updateMode === 'frontend' && otaInfo.partitions} + + ({(otaInfo.partitions.find(p => p.label === otaInfo.target_partition)?.size / 1024 / 1024).toFixed(1)} MB) + + {/if} +
+
+ + {#if status === "success"} +
+ + Update successful! Rebooting device... +
+ {:else} +
{ e.preventDefault(); isDragging = true; }} + ondragleave={() => isDragging = false} + ondrop={handleDrop} + > + +
{updateMode === 'frontend' ? '🎨' : '⚙️'}
+ {#if selectedFile} +
{selectedFile.name}
+
{(selectedFile.size / 1024).toFixed(1)} KB
+ {:else} +
Drop {updateMode === 'bundle' ? 'Universal .bundle' : updateMode === 'frontend' ? 'UI .bin' : 'Firmware .bin'} here
+
or click to browse
+ {/if} +
+ + {#if selectedFile} + + {/if} + + {#if status === "uploading"} +
+
+
+ {:else if status === "error"} +

{errorMsg}

+ {/if} + {/if} +
+ {/if} +
+
+{/if} diff --git a/Provider/frontend/src/lib/api.js b/Provider/frontend/src/lib/api.js index 9424822..72029bb 100644 --- a/Provider/frontend/src/lib/api.js +++ b/Provider/frontend/src/lib/api.js @@ -40,7 +40,7 @@ export async function reboot() { /** * Fetch OTA status from the ESP32. - * @returns {Promise<{active_slot: number, active_partition: string, target_partition: string}>} + * @returns {Promise<{active_slot: number, active_partition: string, target_partition: string, partitions: any[], running_firmware_label: string, running_firmware_slot: number}>} */ export async function getOTAStatus() { const res = await fetch(`${API_BASE}/api/ota/status`); @@ -73,3 +73,46 @@ export async function uploadOTAFrontend(file) { return res.json(); } +/** + * Upload a new firmware binary image. + * @param {File} file The firmware binary file to upload. + * @returns {Promise<{status: string, message: string}>} + */ +export async function uploadOTAFirmware(file) { + const res = await fetch(`${API_BASE}/api/ota/firmware`, { + method: 'POST', + body: file, + headers: { + 'Content-Type': 'application/octet-stream' + } + }); + + if (!res.ok) { + const errorText = await res.text(); + throw new Error(`Upload failed (${res.status}): ${errorText || res.statusText}`); + } + + return res.json(); +} + +/** + * Upload a universal .bundle image (FW + WWW). + * @param {File} file The bundle binary file to upload. + * @returns {Promise<{status: string, message: string}>} + */ +export async function uploadOTABundle(file) { + const res = await fetch(`${API_BASE}/api/ota/bundle`, { + method: 'POST', + body: file, + headers: { + 'Content-Type': 'application/octet-stream' + } + }); + + if (!res.ok) { + const errorText = await res.text(); + throw new Error(`Upload failed (${res.status}): ${errorText || res.statusText}`); + } + + return res.json(); +} diff --git a/Provider/frontend/version.json b/Provider/frontend/version.json index c44a581..9caea60 100644 --- a/Provider/frontend/version.json +++ b/Provider/frontend/version.json @@ -1,5 +1,5 @@ { "major": 0, "minor": 1, - "revision": 7 + "revision": 14 } \ No newline at end of file diff --git a/Provider/main/api/ota/bundle.cpp b/Provider/main/api/ota/bundle.cpp new file mode 100644 index 0000000..c2a4886 --- /dev/null +++ b/Provider/main/api/ota/bundle.cpp @@ -0,0 +1,187 @@ +// SDK +#include "cJSON.h" +#include "esp_http_server.h" +#include "esp_log.h" +#include "esp_ota_ops.h" +#include "esp_partition.h" +#include "esp_system.h" +#include "esp_timer.h" +#include "nvs.h" +#include "nvs_flash.h" +#include + +// Project +#include "appstate.hpp" +#include "types.hpp" + +#define BUNDLE_SCRATCH_BUFSIZE 4096 + +typedef struct +{ + char magic[4]; + uint32_t fw_size; + uint32_t www_size; +} bundle_header_t; + +internal void bundle_ota_restart_timer_callback(void *arg) { esp_restart(); } + +internal esp_err_t api_ota_bundle_handler(httpd_req_t *req) +{ + httpd_resp_set_hdr(req, "Access-Control-Allow-Origin", "*"); + + if (req->content_len < sizeof(bundle_header_t)) + { + ESP_LOGE("OTA_BUNDLE", "Request content too short"); + httpd_resp_send_err(req, HTTPD_400_BAD_REQUEST, "Content too short"); + return ESP_FAIL; + } + + char *buf = (char *)malloc(BUNDLE_SCRATCH_BUFSIZE); + if (!buf) + { + ESP_LOGE("OTA_BUNDLE", "Failed to allocate buffer"); + httpd_resp_send_err(req, HTTPD_500_INTERNAL_SERVER_ERROR, "Out of memory"); + return ESP_FAIL; + } + + // 1. Read Header + bundle_header_t header; + int overhead = httpd_req_recv(req, (char *)&header, sizeof(bundle_header_t)); + if (overhead <= 0) + { + free(buf); + httpd_resp_send_err(req, HTTPD_500_INTERNAL_SERVER_ERROR, + "Header receive failed"); + return ESP_FAIL; + } + + if (memcmp(header.magic, "BNDL", 4) != 0) + { + free(buf); + ESP_LOGE("OTA_BUNDLE", "Invalid magic: %.4s", header.magic); + httpd_resp_send_err(req, HTTPD_400_BAD_REQUEST, "Invalid bundle magic"); + return ESP_FAIL; + } + + ESP_LOGI("OTA_BUNDLE", + "Starting Universal Update: FW %lu bytes, WWW %lu bytes", + header.fw_size, header.www_size); + + // 2. Prepare Firmware Update + const esp_partition_t *fw_part = esp_ota_get_next_update_partition(NULL); + esp_ota_handle_t fw_handle = 0; + esp_err_t err = esp_ota_begin(fw_part, header.fw_size, &fw_handle); + if (err != ESP_OK) + { + free(buf); + ESP_LOGE("OTA_BUNDLE", "esp_ota_begin failed (%s)", esp_err_to_name(err)); + httpd_resp_send_err(req, HTTPD_500_INTERNAL_SERVER_ERROR, + "OTA Begin failed"); + return ESP_FAIL; + } + + // 3. Stream Firmware + uint32_t fw_remaining = header.fw_size; + bool fw_first_chunk = true; + while (fw_remaining > 0) + { + int recv_len = + httpd_req_recv(req, buf, MIN(fw_remaining, BUNDLE_SCRATCH_BUFSIZE)); + if (recv_len <= 0) + { + if (recv_len == HTTPD_SOCK_ERR_TIMEOUT) + continue; + esp_ota_abort(fw_handle); + free(buf); + return ESP_FAIL; + } + + if (fw_first_chunk && recv_len > 0) + { + if ((uint8_t)buf[0] != 0xE9) + { + ESP_LOGE("OTA_BUNDLE", "Invalid FW magic in bundle: %02X", + (uint8_t)buf[0]); + esp_ota_abort(fw_handle); + free(buf); + httpd_resp_send_err( + req, HTTPD_400_BAD_REQUEST, + "Invalid Bundle: Firmware part is corrupted or invalid."); + return ESP_FAIL; + } + fw_first_chunk = false; + } + + esp_ota_write(fw_handle, buf, recv_len); + fw_remaining -= recv_len; + } + esp_ota_end(fw_handle); + + // 4. Prepare WWW Update + uint8_t target_www_slot = g_Active_WWW_Partition == 0 ? 1 : 0; + const char *www_label = target_www_slot == 0 ? "www_0" : "www_1"; + const esp_partition_t *www_part = esp_partition_find_first( + ESP_PARTITION_TYPE_DATA, ESP_PARTITION_SUBTYPE_DATA_LITTLEFS, www_label); + + esp_partition_erase_range(www_part, 0, www_part->size); + + // 5. Stream WWW + uint32_t www_remaining = header.www_size; + uint32_t www_written = 0; + while (www_remaining > 0) + { + int recv_len = + httpd_req_recv(req, buf, MIN(www_remaining, BUNDLE_SCRATCH_BUFSIZE)); + if (recv_len <= 0) + { + if (recv_len == HTTPD_SOCK_ERR_TIMEOUT) + continue; + free(buf); + return ESP_FAIL; + } + esp_partition_write(www_part, www_written, buf, recv_len); + www_written += recv_len; + www_remaining -= recv_len; + } + + free(buf); + + // 6. Commit Updates + esp_ota_set_boot_partition(fw_part); + + nvs_handle_t nvs_h; + if (nvs_open("storage", NVS_READWRITE, &nvs_h) == ESP_OK) + { + nvs_set_u8(nvs_h, "www_part", target_www_slot); + nvs_commit(nvs_h); + nvs_close(nvs_h); + } + + ESP_LOGI("OTA_BUNDLE", "Universal Update Complete! Rebooting..."); + + httpd_resp_set_type(req, "application/json"); + cJSON *root = cJSON_CreateObject(); + cJSON_AddStringToObject(root, "status", "success"); + cJSON_AddStringToObject(root, "message", + "Universal update successful, rebooting..."); + const char *resp = cJSON_Print(root); + httpd_resp_sendstr(req, resp); + free((void *)resp); + cJSON_Delete(root); + + // Reboot + esp_timer_create_args_t tmr_args = {}; + tmr_args.callback = &bundle_ota_restart_timer_callback; + tmr_args.name = "bundle_reboot"; + esp_timer_handle_t tmr; + esp_timer_create(&tmr_args, &tmr); + esp_timer_start_once(tmr, 1'000'000); + + return ESP_OK; +} + +internal const httpd_uri_t api_ota_bundle_uri = {.uri = "/api/ota/bundle", + .method = HTTP_POST, + .handler = + api_ota_bundle_handler, + .user_ctx = NULL}; diff --git a/Provider/main/api/ota/firmware.cpp b/Provider/main/api/ota/firmware.cpp new file mode 100644 index 0000000..cdc6159 --- /dev/null +++ b/Provider/main/api/ota/firmware.cpp @@ -0,0 +1,164 @@ +// SDK +#include "cJSON.h" +#include "esp_http_server.h" +#include "esp_log.h" +#include "esp_ota_ops.h" +#include "esp_system.h" +#include "esp_timer.h" +#include + +// Project +#include "appstate.hpp" +#include "types.hpp" + +#define OTA_FIRMWARE_SCRATCH_BUFSIZE 4096 + +internal void firmware_ota_restart_timer_callback(void *arg) { esp_restart(); } + +internal esp_err_t api_ota_firmware_handler(httpd_req_t *req) +{ + httpd_resp_set_hdr(req, "Access-Control-Allow-Origin", "*"); + + const esp_partition_t *update_partition = + esp_ota_get_next_update_partition(NULL); + if (update_partition == NULL) + { + ESP_LOGE("OTA_FW", "Passive OTA partition not found"); + httpd_resp_send_err(req, HTTPD_500_INTERNAL_SERVER_ERROR, + "OTA partition not found"); + return ESP_FAIL; + } + + ESP_LOGI("OTA_FW", "Writing to partition subtype %d at offset 0x%lx", + update_partition->subtype, update_partition->address); + + esp_ota_handle_t update_handle = 0; + esp_err_t err = + esp_ota_begin(update_partition, OTA_SIZE_UNKNOWN, &update_handle); + if (err != ESP_OK) + { + ESP_LOGE("OTA_FW", "esp_ota_begin failed (%s)", esp_err_to_name(err)); + httpd_resp_send_err(req, HTTPD_500_INTERNAL_SERVER_ERROR, + "OTA begin failed"); + return ESP_FAIL; + } + + char *buf = (char *)malloc(OTA_FIRMWARE_SCRATCH_BUFSIZE); + if (!buf) + { + ESP_LOGE("OTA_FW", "Failed to allocate buffer"); + httpd_resp_send_err(req, HTTPD_500_INTERNAL_SERVER_ERROR, "Out of memory"); + return ESP_FAIL; + } + + int binary_file_len = 0; + int remaining = req->content_len; + + while (remaining > 0) + { + int recv_len = + httpd_req_recv(req, buf, MIN(remaining, OTA_FIRMWARE_SCRATCH_BUFSIZE)); + if (recv_len <= 0) + { + if (recv_len == HTTPD_SOCK_ERR_TIMEOUT) + { + continue; + } + ESP_LOGE("OTA_FW", "Receive failed"); + esp_ota_abort(update_handle); + free(buf); + httpd_resp_send_err(req, HTTPD_500_INTERNAL_SERVER_ERROR, + "Receive failed"); + return ESP_FAIL; + } + + if (binary_file_len == 0 && recv_len > 0) + { + if ((uint8_t)buf[0] != 0xE9) + { + ESP_LOGE("OTA_FW", "Invalid magic: %02X. Expected 0xE9 for Firmware.", + (uint8_t)buf[0]); + esp_ota_abort(update_handle); + free(buf); + httpd_resp_send_err( + req, HTTPD_400_BAD_REQUEST, + "Invalid file: This does not look like an ESP32 firmware binary."); + return ESP_FAIL; + } + } + + err = esp_ota_write(update_handle, (const void *)buf, recv_len); + if (err != ESP_OK) + { + ESP_LOGE("OTA_FW", "esp_ota_write failed (%s)", esp_err_to_name(err)); + esp_ota_abort(update_handle); + free(buf); + httpd_resp_send_err(req, HTTPD_500_INTERNAL_SERVER_ERROR, + "Flash write failed"); + return ESP_FAIL; + } + + binary_file_len += recv_len; + remaining -= recv_len; + } + + free(buf); + ESP_LOGI("OTA_FW", "Total binary data written: %d", binary_file_len); + + err = esp_ota_end(update_handle); + if (err != ESP_OK) + { + if (err == ESP_ERR_OTA_VALIDATE_FAILED) + { + ESP_LOGE("OTA_FW", "Image validation failed, image is corrupted"); + } + else + { + ESP_LOGE("OTA_FW", "esp_ota_end failed (%s)!", esp_err_to_name(err)); + } + httpd_resp_send_err(req, HTTPD_500_INTERNAL_SERVER_ERROR, + "OTA validation/end failed"); + return ESP_FAIL; + } + + err = esp_ota_set_boot_partition(update_partition); + if (err != ESP_OK) + { + ESP_LOGE("OTA_FW", "esp_ota_set_boot_partition failed (%s)!", + esp_err_to_name(err)); + httpd_resp_send_err(req, HTTPD_500_INTERNAL_SERVER_ERROR, + "Failed to set boot partition"); + return ESP_FAIL; + } + + ESP_LOGI("OTA_FW", "OTA successful, rebooting..."); + + httpd_resp_set_type(req, "application/json"); + cJSON *root = cJSON_CreateObject(); + cJSON_AddStringToObject(root, "status", "success"); + cJSON_AddStringToObject(root, "message", + "Firmware update successful, rebooting..."); + const char *response_text = cJSON_Print(root); + httpd_resp_sendstr(req, response_text); + free((void *)response_text); + cJSON_Delete(root); + + // Trigger reboot with 1s delay + const esp_timer_create_args_t restart_timer_args = { + .callback = &firmware_ota_restart_timer_callback, + .arg = (void *)0, + .dispatch_method = ESP_TIMER_TASK, + .name = "fw_ota_restart_timer", + .skip_unhandled_events = false}; + esp_timer_handle_t restart_timer; + esp_timer_create(&restart_timer_args, &restart_timer); + esp_timer_start_once(restart_timer, 1'000'000); + + return ESP_OK; +} + +internal const httpd_uri_t api_ota_firmware_uri = {.uri = "/api/ota/firmware", + .method = HTTP_POST, + .handler = + api_ota_firmware_handler, + .user_ctx = NULL}; diff --git a/Provider/main/api/ota/frontend.cpp b/Provider/main/api/ota/frontend.cpp index fc5ed90..dfa5b27 100644 --- a/Provider/main/api/ota/frontend.cpp +++ b/Provider/main/api/ota/frontend.cpp @@ -8,7 +8,6 @@ #include "nvs.h" #include "nvs_flash.h" - // Project #include "appstate.hpp" #include "types.hpp" @@ -59,6 +58,7 @@ internal esp_err_t api_ota_frontend_handler(httpd_req_t *req) int total_read = 0; int remaining = req->content_len; + bool first_chunk = true; while (remaining > 0) { @@ -77,6 +77,21 @@ internal esp_err_t api_ota_frontend_handler(httpd_req_t *req) return ESP_FAIL; } + if (first_chunk) + { + if ((uint8_t)buf[0] == 0xE9) + { + ESP_LOGE("OTA", "Magic 0xE9 detected. This looks like a FIRMWARE bin, " + "but you are uploading to FRONTEND slot!"); + free(buf); + httpd_resp_send_err( + req, HTTPD_400_BAD_REQUEST, + "Invalid file: This is a Firmware binary, not a UI binary."); + return ESP_FAIL; + } + first_chunk = false; + } + err = esp_partition_write(partition, total_read, buf, recv_len); if (err != ESP_OK) { diff --git a/Provider/main/api/ota/status.cpp b/Provider/main/api/ota/status.cpp index 8da18b0..2570b99 100644 --- a/Provider/main/api/ota/status.cpp +++ b/Provider/main/api/ota/status.cpp @@ -1,12 +1,20 @@ +#include + // SDK #include "cJSON.h" #include "esp_http_server.h" +#include "esp_image_format.h" #include "esp_littlefs.h" +#include "esp_log.h" +#include "esp_ota_ops.h" #include "esp_partition.h" +#include "esp_vfs.h" +#include // Project #include "appstate.hpp" #include "types.hpp" +#include "utils.hpp" internal esp_err_t api_ota_status_handler(httpd_req_t *req) { @@ -17,34 +25,58 @@ internal esp_err_t api_ota_status_handler(httpd_req_t *req) cJSON_AddNumberToObject(root, "active_slot", g_Active_WWW_Partition); - const char *partitions[] = {"www_0", "www_1"}; cJSON *parts_arr = cJSON_AddArrayToObject(root, "partitions"); - for (int i = 0; i < 2; i++) + esp_partition_iterator_t it = esp_partition_find( + ESP_PARTITION_TYPE_ANY, ESP_PARTITION_SUBTYPE_ANY, NULL); + while (it != NULL) { + const esp_partition_t *p = esp_partition_get(it); cJSON *p_obj = cJSON_CreateObject(); - cJSON_AddStringToObject(p_obj, "label", partitions[i]); + cJSON_AddStringToObject(p_obj, "label", p->label); + cJSON_AddNumberToObject(p_obj, "type", p->type); + cJSON_AddNumberToObject(p_obj, "subtype", p->subtype); + cJSON_AddNumberToObject(p_obj, "address", p->address); + cJSON_AddNumberToObject(p_obj, "size", p->size); - const esp_partition_t *p = esp_partition_find_first( - ESP_PARTITION_TYPE_DATA, ESP_PARTITION_SUBTYPE_ANY, partitions[i]); - if (p) + // Try to get LittleFS info if it's a data partition + if (p->type == ESP_PARTITION_TYPE_DATA) { - cJSON_AddNumberToObject(p_obj, "size", p->size); - size_t total = 0, used = 0; - if (esp_littlefs_info(partitions[i], &total, &used) == ESP_OK) + if (esp_littlefs_info(p->label, &total, &used) == ESP_OK) { cJSON_AddNumberToObject(p_obj, "used", used); cJSON_AddNumberToObject(p_obj, "free", total - used); } else { - // Not mounted or not LFS - cJSON_AddNumberToObject(p_obj, "used", 0); - cJSON_AddNumberToObject(p_obj, "free", p->size); + // For other data partitions (nvs, phy_init), just show total as used + // for now + cJSON_AddNumberToObject(p_obj, "used", p->size); + cJSON_AddNumberToObject(p_obj, "free", 0); } } + // For app partitions, try to find the binary size + else if (p->type == ESP_PARTITION_TYPE_APP) + { + esp_app_desc_t app_desc; + if (esp_ota_get_partition_description(p, &app_desc) == ESP_OK) + { + cJSON_AddStringToObject(p_obj, "app_version", app_desc.version); + + // Get the true binary size from image metadata + esp_image_metadata_t data; + const esp_partition_pos_t pos = {.offset = p->address, .size = p->size}; + if (esp_image_get_metadata(&pos, &data) == ESP_OK) + { + cJSON_AddNumberToObject(p_obj, "used", data.image_len); + cJSON_AddNumberToObject(p_obj, "free", p->size - data.image_len); + } + } + } + cJSON_AddItemToArray(parts_arr, p_obj); + it = esp_partition_next(it); } cJSON_AddStringToObject(root, "active_partition", @@ -52,6 +84,24 @@ internal esp_err_t api_ota_status_handler(httpd_req_t *req) cJSON_AddStringToObject(root, "target_partition", g_Active_WWW_Partition == 0 ? "www_1" : "www_0"); + const esp_partition_t *running = esp_ota_get_running_partition(); + if (running) + { + cJSON_AddStringToObject(root, "running_firmware_label", running->label); + if (running->subtype >= ESP_PARTITION_SUBTYPE_APP_OTA_MIN && + running->subtype <= ESP_PARTITION_SUBTYPE_APP_OTA_MAX) + { + cJSON_AddNumberToObject(root, "running_firmware_slot", + running->subtype - + ESP_PARTITION_SUBTYPE_APP_OTA_MIN); + } + else + { + cJSON_AddNumberToObject(root, "running_firmware_slot", + -1); // Factory or other + } + } + const char *status_info = cJSON_Print(root); httpd_resp_sendstr(req, status_info); diff --git a/Provider/main/http_server.cpp b/Provider/main/http_server.cpp index 1877a05..7eb87b8 100644 --- a/Provider/main/http_server.cpp +++ b/Provider/main/http_server.cpp @@ -13,12 +13,13 @@ #endif // Project +#include "api/ota/bundle.cpp" +#include "api/ota/firmware.cpp" #include "api/ota/frontend.cpp" #include "api/ota/status.cpp" #include "api/system/info.cpp" #include "api/system/reboot.cpp" - internal const char *TAG = "HTTP_SERVER"; constexpr uint8 kGZ_Extension_Length = sizeof(".gz") - 1; @@ -234,6 +235,8 @@ internal httpd_handle_t start_webserver(void) httpd_register_uri_handler(server, &api_system_reboot_uri); httpd_register_uri_handler(server, &api_ota_status_uri); httpd_register_uri_handler(server, &api_ota_frontend_uri); + httpd_register_uri_handler(server, &api_ota_firmware_uri); + httpd_register_uri_handler(server, &api_ota_bundle_uri); #ifdef CONFIG_CALENDINK_DEPLOY_WEB_PAGES // Register static file handler last as a catch-all wildcard if deployed diff --git a/Provider/main/main.cpp b/Provider/main/main.cpp index 31d033b..a20e95a 100644 --- a/Provider/main/main.cpp +++ b/Provider/main/main.cpp @@ -3,14 +3,14 @@ // SDK #include "esp_log.h" +#include "esp_ota_ops.h" +#include "esp_system.h" #include "freertos/FreeRTOS.h" #include "freertos/task.h" #include "nvs.h" #include "nvs_flash.h" -#include "sdkconfig.h" #include "soc/gpio_num.h" - // Project headers #include "appstate.hpp" #include "types.hpp" @@ -26,7 +26,7 @@ internal constexpr bool kBlockUntilEthernetEstablished = false; extern "C" void app_main() { - printf("Hello, Worldi!\n"); + printf("Hello, Calendink OTA! [V1.1]\n"); httpd_handle_t web_server = NULL; @@ -41,11 +41,27 @@ extern "C" void app_main() nvs_handle_t my_handle; if (nvs_open("storage", NVS_READWRITE, &my_handle) == ESP_OK) { - err = nvs_get_u8(my_handle, "www_part", &g_Active_WWW_Partition); - if (err != ESP_OK && err != ESP_ERR_NVS_NOT_FOUND) + // If we are running from the factory partition, force the www partition to + // 0 This ensures that after a USB flash (which only writes to www_0), we + // aren't stuck looking at an old www_1. + const esp_partition_t *running = esp_ota_get_running_partition(); + if (running && running->subtype == ESP_PARTITION_SUBTYPE_APP_FACTORY) { - printf("Error reading www_part from NVS: %s\n", esp_err_to_name(err)); + printf( + "Running from factory: resetting www_part to 0 for consistency.\n"); + g_Active_WWW_Partition = 0; + nvs_set_u8(my_handle, "www_part", 0); + nvs_commit(my_handle); } + else + { + err = nvs_get_u8(my_handle, "www_part", &g_Active_WWW_Partition); + if (err != ESP_OK && err != ESP_ERR_NVS_NOT_FOUND) + { + printf("Error reading www_part from NVS: %s\n", esp_err_to_name(err)); + } + } + if (g_Active_WWW_Partition > 1) { g_Active_WWW_Partition = 0; @@ -151,6 +167,9 @@ extern "C" void app_main() printf("Connected!\n"); + // Mark the current app as valid to cancel rollback + esp_ota_mark_app_valid_cancel_rollback(); + // Start the webserver web_server = start_webserver(); diff --git a/Provider/main/utils.hpp b/Provider/main/utils.hpp new file mode 100644 index 0000000..111975f --- /dev/null +++ b/Provider/main/utils.hpp @@ -0,0 +1,8 @@ +#pragma once + +#include + +template constexpr size_t ArrayCount(T (&)[N]) +{ + return N; +} \ No newline at end of file diff --git a/Provider/partitions.csv b/Provider/partitions.csv index 9c44e65..5039963 100644 --- a/Provider/partitions.csv +++ b/Provider/partitions.csv @@ -1,6 +1,9 @@ # Name, Type, SubType, Offset, Size, Flags nvs, data, nvs, 0x9000, 0x6000, -phy_init, data, phy, 0xf000, 0x1000, -factory, app, factory, 0x10000, 1M, +otadata, data, ota, , 0x2000, +phy_init, data, phy, , 0x1000, +factory, app, factory, , 2M, +ota_0, app, ota_0, , 2M, +ota_1, app, ota_1, , 2M, www_0, data, littlefs, , 1M, www_1, data, littlefs, , 1M, diff --git a/Provider/tdd/firmware_ota.md b/Provider/tdd/firmware_ota.md new file mode 100644 index 0000000..4d3059f --- /dev/null +++ b/Provider/tdd/firmware_ota.md @@ -0,0 +1,79 @@ +# Firmware OTA Strategy for ESP32-S3 Provider + +**Authored by Antigravity** +**Date:** 2026-03-03 + +--- + +## 1. Goal + +Implement a robust Over-The-Air (OTA) update mechanism for both the main firmware of the ESP32-S3 and the Svelte frontend. The update must: +- Update the core application logic and the user interface without requiring a physical USB connection. +- Keep the Firmware and Frontend in sync by allowing them to be updated together atomically. +- Provide a reliable fallback if an update fails (Rollback capability via A/B slots). +- Provide a permanent "factory" fallback as an extreme safety measure. +- Prevent accidental cross-flashing (e.g., flashing UI to firmware slots). +- Maintain a clear versioning scheme visible to the user, with accurate partition space reporting. + +## 2. Chosen Approach + +We implemented a **Universal Dual-Partition OTA system** using ESP-IDF's native OTA mechanisms for the firmware and LittleFS for the frontend. + +Updates can be performed individually (Firmware only via `.bin`, Frontend only via `.bin`), but the primary and recommended approach is the **Universal OTA Bundle**. +The build process generates a single `.bundle` file containing both the firmware image and the compiled frontend filesystem. This bundle is uploaded via the frontend UI, streamed directly to the inactive OTA flash partition (`ota_0` or `ota_1`) and inactive UI partition (`www_0` or `www_1`). Upon successful transfer and validation of both components, the bootloader and NVS are instructed to switch active partitions on the next restart. + +## 3. Design Decisions & Trade-offs + +### 3.1. Why Dual-Partition (A/B) with Factory? +- **Safety**: A failed or interrupted upload never "bricks" the device. +- **Factory Fallback**: By maintaining a dedicated 2MB `factory` partition alongside the two 2MB OTA partitions (`ota_0`, `ota_1`), we ensure that even if both OTA slots are irrecoverably corrupted, the device can always boot into a known-good state. +- **Frontend Sync**: The frontend also uses a dual-partition layout (`www_0`, `www_1`). The Universal Bundle ensures both FW and UI switch together. + +### 3.2. Automatic App Rollback +We rely on ESP-IDF's built-in "App Rollback" feature. +- **The Mechanism**: When the ESP32 boots a newly OTA-flashed firmware, it is marked as "Pending Verify". If the application crashes or fails to mark itself as "valid", the bootloader reverts to the previous working partition. +- **Validation Point**: We consider the firmware "valid" only after it successfully establishes a network connection. + +### 3.3. Universal Bundle Format & Automation +- **Format**: A custom 12-byte header (`BNDL` magic + 4-byte FW size + 4-byte UI size) followed by the FW binary and UI binary. +- **Automation**: The Svelte build chain automates packaging. Running `npm run ota:bundle` automatically triggers Vite production build, LittleFS frontend packaging, applies proper semantic version sorting (to always pick the latest compiled UI), and generates the `.bundle` payload. + +### 3.4. Safety & Validation +- **Magic Number Checks**: The backend enforces strict validation before writing to flash. Firmware endpoints and bundle streams check for the ESP32 image magic byte (`0xE9`), and Bundle endpoints check for the `BNDL` magic header. This prevents a user from accidentally uploading a LittleFS image to the Firmware slot, avoiding immediate boot loops. +- **Atomic Commits**: The Universal Bundle handler only sets the new boot partition and updates the NVS UI partition index if *both* firmware and frontend streams complete successfully. + +### 3.5. Versioning & Partition Metadata +- **Firmware Versioning**: Extracted natively from `esp_app_desc_t`, syncing API version with CMake `PROJECT_VER`. +- **Space Reporting**: The system dynamically scans App partitions using `esp_image_get_metadata()` to determine the exact binary size flashed in each slot. This allows the UI to display accurate "used" and "free" space per partition, regardless of the fixed partition size. + +## 4. Final Architecture + +### 4.1. The Partition Table +```csv +# Name, Type, SubType, Offset, Size +nvs, data, nvs, 0x9000, 0x6000 +otadata, data, ota, , 0x2000 +phy_init, data, phy, , 0x1000 +factory, app, factory, , 2M +ota_0, app, ota_0, , 2M +ota_1, app, ota_1, , 2M +www_0, data, littlefs, , 1M +www_1, data, littlefs, , 1M +``` + +### 4.2. Backend Components +- `bundle.cpp`: Handles `POST /api/ota/bundle`. Streams the file, splitting it on the fly into the inactive `ota` and `www` partitions. +- `firmware.cpp` & `frontend.cpp`: Handles individual component updates. +- `status.cpp`: Uses `esp_partition_find` and `esp_image_get_metadata` to report partition sizes and active slots. +- `main.cpp`: Calls `esp_ota_mark_app_valid_cancel_rollback()` post-network connection and manages NVS synchronization for the UI slot when booting from Factory. + +### 4.3. UI/UX Implementation +- The Svelte Dashboard features a comprehensive "Update System" component supporting individual (FW/UI) and combined (Bundle) uploads. +- A "Partition Table" view provides real-time visibility into the exact binary size, available free space, and version hash of every system and app partition. + +## 5. Summary + +We use **ESP-IDF's native OTA APIs** with a **Factory + Dual A/B Partition** layout, synchronized with a **Dual LittleFS Partition** layout for the frontend. The system relies on custom **Universal Bundles** to guarantee atomic FW+UI upgrades, protected by **Magic Number validations** and **Automatic App Rollbacks**. The entire process is driven from a highly integrated Svelte UI that leverages backend metadata extraction to provide accurate system insights. + +--- +*Created by Antigravity - Last Updated: 2026-03-03*