Calendink/Provider/tdd/backend_architecture.md

Backend Architecture for ESP32-S3 Provider

Authored by Claude Opus 4 Date: 2026-03-02

---

1. Goal

Serve the Svelte web dashboard and expose a REST API from the ESP32-S3 using ESP-IDF's built-in esp_http_server. The backend must:

• Serve static files (the compiled Svelte frontend) from flash
• Provide system information over JSON
• Allow remote reboot
• Support independent frontend/backend development workflows

2. Chosen Stack

Technology	Source	Role
esp_http_server	ESP-IDF built-in	HTTP server daemon
cJSON	ESP-IDF built-in	JSON serialization for API responses
LittleFS	joltwallet/esp_littlefs	Filesystem on flash for serving frontend files

All three are standard in ESP-IDF projects. esp_http_server and cJSON ship with the SDK. LittleFS is the recommended flash filesystem for ESP-IDF — Espressif's own restful_server example uses joltwallet/littlefs.

3. Why LittleFS Over Other Options

LittleFS vs SPIFFS

	LittleFS	SPIFFS
Directory support	✅ Real directories	❌ Flat namespace
Wear leveling	✅ Dynamic	⚠️ Static
Power-loss safe	✅ Copy-on-write	❌ Can corrupt
ESP-IDF status	✅ Recommended	⚠️ Deprecated in recent examples
Performance	Faster for small files	Slower mount, no dir listing

SPIFFS was the original choice in older ESP-IDF examples but has been replaced by LittleFS in the current restful_server example. LittleFS is the better choice going forward.

LittleFS Partition vs EMBED_FILES

We considered two approaches to deploy the frontend:

	LittleFS Partition	EMBED_FILES (binary embedding)
Storage	Separate flash partition	Compiled into firmware binary
Update	Can flash partition independently	Must reflash entire firmware
OTA	Needs separate partition OTA	UI updates with firmware naturally
Dev workflow	Can skip frontend flash during iteration	Always included
Filesystem	Real VFS — open/read/close file APIs	Direct memory pointer

We chose LittleFS partition because:

1. Development speed — a Kconfig toggle (CALENDINK_DEPLOY_WEB_PAGES) lets you skip flashing the frontend entirely when iterating on the backend
2. Separation of concerns — frontend and firmware have independent flash regions
3. Follows ESP-IDF conventions — matches the official restful_server example pattern
4. No RAM overhead — files are read from flash in chunks, not loaded into memory

4. Kconfig Deploy Toggle

The CALENDINK_DEPLOY_WEB_PAGES menuconfig option controls whether the frontend gets flashed:

Setting	Effect	Use case
OFF (default)	Frontend not flashed. API still works.	Backend development — fast flash, use PC dev server for UI
ON	frontend/dist/ is written to the www LittleFS partition	Production deployment — everything runs on ESP32

This mirrors the official ESP-IDF pattern (CONFIG_EXAMPLE_DEPLOY_WEB_PAGES in the restful_server example).

5. API Design

Endpoints

GET  /api/system/info    → JSON with chip, heap, uptime, firmware, connection
POST /api/system/reboot  → JSON acknowledgment, then esp_restart()
GET  /*                   → Static files from LittleFS /www (when deployed)

CORS

During development, the Svelte dev server runs on http://localhost:5173 and API calls go to http://<ESP32_IP>. This is cross-origin, so the backend adds CORS headers:

• Access-Control-Allow-Origin: *
• Access-Control-Allow-Methods: GET, POST, OPTIONS
• Access-Control-Allow-Headers: Content-Type
• OPTIONS preflight handler

In production (frontend served from ESP32), everything is same-origin — CORS headers have no effect but don't hurt.

6. File Organization

Provider/main/
├── main.cpp                    # Entry point, network init, starts HTTP server
├── http_server.cpp             # HTTP server lifecycle, static file handler, CORS
├── api/
│   └── system/
│       ├── info.cpp            # GET /api/system/info
│       └── reboot.cpp          # POST /api/system/reboot
├── led_status.cpp              # Existing — LED management
├── connect.cpp                 # Existing — Ethernet/WiFi
├── types.hpp                   # Existing — type aliases
└── Kconfig.projbuild           # Existing + new web server config

Why This Structure

The project uses a unity build pattern — main.cpp #includes .cpp files directly (e.g. #include "connect.cpp"). This is unconventional but works well for small embedded projects since ESP-IDF only compiles the files listed in idf_component_register(SRCS ...).

We extend this pattern to the HTTP server:

• http_server.cpp #includes the API handler files
• Each handler file is self-contained: it defines its URI struct and registration function
• New endpoint groups get their own folder under api/ (e.g. api/calendar/, api/display/)

7. Partition Table

# Name,   Type, SubType,  Offset,   Size
nvs,      data, nvs,      0x9000,   0x6000
phy_init, data, phy,      0xf000,   0x1000
factory,  app,  factory,  0x10000,  1M
www,      data, littlefs, ,         64K

The www partition is 64KB — more than enough for the 16kB gzipped frontend. Only gets written during idf.py flash when CALENDINK_DEPLOY_WEB_PAGES is enabled.

8. Build Pipeline

Frontend Build (PC)                      ESP-IDF Build
──────────────────                       ──────────────
npm run build                             idf.py build
  ↓                                         ↓
frontend/dist/                           firmware.bin
  index.html (47kB)                          +
  index.html.gz (16kB)                   www.bin (LittleFS image, when deploy=ON)
                                              ↓
                                         idf.py flash
                                              ↓
                                         ESP32-S3 Flash
                                         ├── factory partition → firmware
                                         └── www partition → frontend files

9. Summary

We use esp_http_server + cJSON + LittleFS — all standard ESP-IDF components — to serve the frontend and expose a REST API. A LittleFS partition stores frontend files separately from firmware, with a Kconfig toggle to skip frontend flashing during backend development. The API is structured as modular handler files under api/ for clean scalability.