Added lvgl support to generate images. Made basic example, grayscale background + text displayed everytime we call /api/display/image.png
This commit is contained in:
75
Provider/tdd/lvgl_image_generation.md
Normal file
75
Provider/tdd/lvgl_image_generation.md
Normal file
@@ -0,0 +1,75 @@
|
||||
# LVGL Image Generation Architecture
|
||||
|
||||
**Authored by Antigravity**
|
||||
**Date:** 2026-03-14
|
||||
|
||||
---
|
||||
|
||||
## 1. Goal
|
||||
|
||||
Integrate the Light and Versatile Graphics Library (LVGL) into the Calendink Provider to generate UI images server-side. The ESP32-S3 will render screens into a memory buffer and serve the resulting image (PNG) directly over the HTTP REST API to clients.
|
||||
|
||||
This allows the Provider to act as a "smart renderer" for dump clients like e-ink displays or web widgets that only have the capability to fetch and display static images. The display resolution (default 800x480) and color depth (4-level grayscale) are parameterized via Kconfig to support future hardware changes.
|
||||
|
||||
## 2. Chosen Approach
|
||||
|
||||
### 2.1. Headless LVGL Display Driver
|
||||
LVGL is typically used to drive physical displays via SPI/I2C/RGB interfaces. For this use case, we will configure a **virtual (headless) display**:
|
||||
- A display instance (`lv_display_t`) will be created without a physical flush callback (or a dummy one that does nothing).
|
||||
- We will allocate a full-screen frame buffer (e.g., 800x480 at 8-bit grayscale). While small enough to fit in SRAM (~384 KB), we will use **PSRAM** to avoid memory pressure on the system.
|
||||
- When an image is requested, we will force LVGL to update the screen (`lv_refr_now()`) and then read the raw pixel data directly from the draw buffer.
|
||||
|
||||
### 2.2. Image Encoding (PNG)
|
||||
Raw pixel data is not web-friendly. To serve the image over HTTP, we will encode it as a **PNG** image.
|
||||
- **Why PNG?** PNG is smaller over the network than BMP and natively supports lossless grayscale compression. Given the target of 4-level grayscale, a PNG will compress exceptionally well.
|
||||
- **Encoder:** We will use `lodepng`, a lightweight single-file C library, to compress the raw LVGL buffer into a PNG on the fly.
|
||||
- **Color Depth:** The target is **4-level grayscale**. We will map LVGL's output to a 2-bit grayscale PNG to minimize the payload size.
|
||||
- **Parameterization:** Resolution (e.g., 800 width, 480 height) is configurable via Kconfig (`CONFIG_DISPLAY_WIDTH`, `CONFIG_DISPLAY_HEIGHT`) so it can be easily changed for different e-ink displays.
|
||||
|
||||
### 2.3. Thread Safety and Concurrency
|
||||
LVGL is **not thread-safe**. Since the HTTP server runs its API handlers in different FreeRTOS tasks (from the `httpd` thread pool), we must protect LVGL with a **Mutex**.
|
||||
- All LVGL initialization, UI setup (`lv_obj_create`, etc.), and the periodic `lv_timer_handler()` will take the `g_LvglMutex`.
|
||||
- The `GET /api/display/image` API endpoint will acquire `g_LvglMutex`, draw the screen, encode the BMP header, transmit the buffer, and then release the mutex.
|
||||
|
||||
## 3. Architecture
|
||||
|
||||
### 3.1. File Structure
|
||||
|
||||
```
|
||||
Provider/main/
|
||||
├── lv_setup.hpp # LVGL initialization and mutex declarations
|
||||
├── lv_setup.cpp # Driver setup, PSRAM buffer allocation, LVGL tick task
|
||||
├── api/display/
|
||||
│ ├── image.cpp # GET /api/display/image handler (BMP streamer)
|
||||
│ └── unity.cpp # Aggregator for display endpoints
|
||||
├── http_server.cpp # Modified to include api/display/unity.cpp
|
||||
└── main.cpp # Starts LVGL task before HTTP server
|
||||
```
|
||||
|
||||
### 3.2. Data Flow
|
||||
|
||||
1. **Client** makes a `GET /api/display/image` request.
|
||||
2. **`image.cpp` handler** intercepts the request and takes the `g_LvglMutex`.
|
||||
3. The handler forces LVGL to finish any pending rendering.
|
||||
4. The handler uses `lodepng` to compress the raw frame buffer into a PNG payload.
|
||||
5. The handler sends the PNG via `httpd_resp_send()`.
|
||||
6. The handler frees the PNG payload buffer and releases `g_LvglMutex`.
|
||||
7. The HTTP response completes.
|
||||
|
||||
## 4. Design Decisions & Trade-offs
|
||||
|
||||
| Decision | Trade-off | Rationale |
|
||||
|---|---|---|
|
||||
| **PSRAM Allocation** | Slower access than SRAM | A full framebuffer for high-res displays exceeds internal SRAM. PSRAM is required, and since we only read/write it occasionally for HTTP, the speed penalty is negligible compared to network latency. |
|
||||
| **PNG over BMP** | More CPU overhead | PNG compression takes CPU cycles and temporary RAM, but significantly reduces network payload, which is better for web clients and saves transmission time. |
|
||||
| **Pull vs Push** | Requires polling | The client must actively request the image. This is standard for web, but means the client won't know instantly if the UI state changes unless using WebSockets/SSE (future scope). |
|
||||
| **Synchronous Render** | Blocks HTTP task | The HTTP handler waits for LVGL to finish drawing and PNG encoding to complete. Since LVGL renders very quickly in memory, this block should be acceptable. |
|
||||
|
||||
## 5. Next Steps
|
||||
1. Add `lvgl/lvgl` to `idf_component.yml`.
|
||||
2. Configure LVGL in `sdkconfig` to use custom memory allocation (routed to PSRAM).
|
||||
3. Implement `lv_setup.cpp` and `api/display/image.cpp`.
|
||||
4. Draw a sample UI on the virtual display.
|
||||
|
||||
---
|
||||
*Created by Antigravity - Last Updated: 2026-03-14*
|
||||
Reference in New Issue
Block a user