From bba4c63f9364c80e9a442eb34e7bf02438e28e2e Mon Sep 17 00:00:00 2001 From: Patedam Date: Mon, 2 Mar 2026 22:21:52 -0500 Subject: [PATCH] docs: Add backend architecture documentation for the ESP32-S3 provider. --- Provider/tdd/backend_architecture.md | 147 +++++++++++++++++++++++++++ 1 file changed, 147 insertions(+) create mode 100644 Provider/tdd/backend_architecture.md diff --git a/Provider/tdd/backend_architecture.md b/Provider/tdd/backend_architecture.md new file mode 100644 index 0000000..b91a23a --- /dev/null +++ b/Provider/tdd/backend_architecture.md @@ -0,0 +1,147 @@ +# Backend Architecture for ESP32-S3 Provider + +**Authored by Claude Opus 4** +**Date:** 2026-03-02 + +--- + +## 1. Goal + +Serve the Svelte web dashboard and expose a REST API from the ESP32-S3 using ESP-IDF's built-in `esp_http_server`. The backend must: +- Serve static files (the compiled Svelte frontend) from flash +- Provide system information over JSON +- Allow remote reboot +- Support independent frontend/backend development workflows + +## 2. Chosen Stack + +| Technology | Source | Role | +|---|---|---| +| **esp_http_server** | ESP-IDF built-in | HTTP server daemon | +| **cJSON** | ESP-IDF built-in | JSON serialization for API responses | +| **LittleFS** | [joltwallet/esp_littlefs](https://github.com/joltwallet/esp_littlefs) | Filesystem on flash for serving frontend files | + +All three are standard in ESP-IDF projects. `esp_http_server` and `cJSON` ship with the SDK. LittleFS is the recommended flash filesystem for ESP-IDF — Espressif's own [restful_server example](https://github.com/espressif/esp-idf/blob/master/examples/protocols/http_server/restful_server/main/idf_component.yml) uses `joltwallet/littlefs`. + +## 3. Why LittleFS Over Other Options + +### LittleFS vs SPIFFS + +| | LittleFS | SPIFFS | +|---|---|---| +| **Directory support** | ✅ Real directories | ❌ Flat namespace | +| **Wear leveling** | ✅ Dynamic | ⚠️ Static | +| **Power-loss safe** | ✅ Copy-on-write | ❌ Can corrupt | +| **ESP-IDF status** | ✅ Recommended | ⚠️ Deprecated in recent examples | +| **Performance** | Faster for small files | Slower mount, no dir listing | + +SPIFFS was the original choice in older ESP-IDF examples but has been replaced by LittleFS in the current restful_server example. LittleFS is the better choice going forward. + +### LittleFS Partition vs EMBED_FILES + +We considered two approaches to deploy the frontend: + +| | LittleFS Partition | `EMBED_FILES` (binary embedding) | +|---|---|---| +| **Storage** | Separate flash partition | Compiled into firmware binary | +| **Update** | Can flash partition independently | Must reflash entire firmware | +| **OTA** | Needs separate partition OTA | UI updates with firmware naturally | +| **Dev workflow** | Can skip frontend flash during iteration | Always included | +| **Filesystem** | Real VFS — open/read/close file APIs | Direct memory pointer | + +**We chose LittleFS partition** because: +1. **Development speed** — a Kconfig toggle (`CALENDINK_DEPLOY_WEB_PAGES`) lets you skip flashing the frontend entirely when iterating on the backend +2. **Separation of concerns** — frontend and firmware have independent flash regions +3. **Follows ESP-IDF conventions** — matches the official restful_server example pattern +4. **No RAM overhead** — files are read from flash in chunks, not loaded into memory + +## 4. Kconfig Deploy Toggle + +The `CALENDINK_DEPLOY_WEB_PAGES` menuconfig option controls whether the frontend gets flashed: + +| Setting | Effect | Use case | +|---|---|---| +| **OFF** (default) | Frontend not flashed. API still works. | Backend development — fast flash, use PC dev server for UI | +| **ON** | `frontend/dist/` is written to the `www` LittleFS partition | Production deployment — everything runs on ESP32 | + +This mirrors the official ESP-IDF pattern (`CONFIG_EXAMPLE_DEPLOY_WEB_PAGES` in the restful_server example). + +## 5. API Design + +### Endpoints + +``` +GET /api/system/info → JSON with chip, heap, uptime, firmware, connection +POST /api/system/reboot → JSON acknowledgment, then esp_restart() +GET /* → Static files from LittleFS /www (when deployed) +``` + +### CORS + +During development, the Svelte dev server runs on `http://localhost:5173` and API calls go to `http://`. This is cross-origin, so the backend adds CORS headers: + +- `Access-Control-Allow-Origin: *` +- `Access-Control-Allow-Methods: GET, POST, OPTIONS` +- `Access-Control-Allow-Headers: Content-Type` +- `OPTIONS` preflight handler + +In production (frontend served from ESP32), everything is same-origin — CORS headers have no effect but don't hurt. + +## 6. File Organization + +``` +Provider/main/ +├── main.cpp # Entry point, network init, starts HTTP server +├── http_server.cpp # HTTP server lifecycle, static file handler, CORS +├── api/ +│ └── system/ +│ ├── info.cpp # GET /api/system/info +│ └── reboot.cpp # POST /api/system/reboot +├── led_status.cpp # Existing — LED management +├── connect.cpp # Existing — Ethernet/WiFi +├── types.hpp # Existing — type aliases +└── Kconfig.projbuild # Existing + new web server config +``` + +### Why This Structure + +The project uses a **unity build** pattern — `main.cpp` `#include`s `.cpp` files directly (e.g. `#include "connect.cpp"`). This is unconventional but works well for small embedded projects since ESP-IDF only compiles the files listed in `idf_component_register(SRCS ...)`. + +We extend this pattern to the HTTP server: +- `http_server.cpp` `#include`s the API handler files +- Each handler file is self-contained: it defines its URI struct and registration function +- New endpoint groups get their own folder under `api/` (e.g. `api/calendar/`, `api/display/`) + +## 7. Partition Table + +``` +# Name, Type, SubType, Offset, Size +nvs, data, nvs, 0x9000, 0x6000 +phy_init, data, phy, 0xf000, 0x1000 +factory, app, factory, 0x10000, 1M +www, data, littlefs, , 64K +``` + +The `www` partition is 64KB — more than enough for the 16kB gzipped frontend. Only gets written during `idf.py flash` when `CALENDINK_DEPLOY_WEB_PAGES` is enabled. + +## 8. Build Pipeline + +``` +Frontend Build (PC) ESP-IDF Build +────────────────── ────────────── +npm run build idf.py build + ↓ ↓ +frontend/dist/ firmware.bin + index.html (47kB) + + index.html.gz (16kB) www.bin (LittleFS image, when deploy=ON) + ↓ + idf.py flash + ↓ + ESP32-S3 Flash + ├── factory partition → firmware + └── www partition → frontend files +``` + +## 9. Summary + +We use **esp_http_server + cJSON + LittleFS** — all standard ESP-IDF components — to serve the frontend and expose a REST API. A **LittleFS partition** stores frontend files separately from firmware, with a **Kconfig toggle** to skip frontend flashing during backend development. The API is structured as **modular handler files** under `api/` for clean scalability.