LVGL Image Generation Architecture

Authored by Antigravity Date: 2026-03-14

1. Goal

Integrate the Light and Versatile Graphics Library (LVGL) into the Calendink Provider to generate UI images server-side. The ESP32-S3 will render screens into a memory buffer and serve the resulting image (PNG) directly over the HTTP REST API to clients.

This allows the Provider to act as a "smart renderer" for dump clients like e-ink displays or web widgets that only have the capability to fetch and display static images. The display resolution (default 800x480) and color depth (4-level grayscale) are parameterized via Kconfig to support future hardware changes.

2. Chosen Approach

2.1. Headless LVGL Display Driver

LVGL is typically used to drive physical displays via SPI/I2C/RGB interfaces. For this use case, we will configure a virtual (headless) display:

A display instance (lv_display_t) will be created without a physical flush callback (or a dummy one that does nothing).
We will allocate a full-screen frame buffer (e.g., 800x480 at 8-bit grayscale). While small enough to fit in SRAM (~384 KB), we will use PSRAM to avoid memory pressure on the system.
When an image is requested, we will force LVGL to update the screen (lv_refr_now()) and then read the raw pixel data directly from the draw buffer.

2.2. Image Encoding (PNG)

Raw pixel data is not web-friendly. To serve the image over HTTP, we will encode it as a PNG image.

Why PNG? PNG is smaller over the network than BMP and natively supports lossless grayscale compression. Given the target of 4-level grayscale, a PNG will compress exceptionally well.
Encoder: We will use lodepng, a lightweight single-file C library, to compress the raw LVGL buffer into a PNG on the fly.
Color Depth: The target is 4-level grayscale. We will map LVGL's output to a 2-bit grayscale PNG to minimize the payload size.
Parameterization: Resolution (e.g., 800 width, 480 height) is configurable via Kconfig (CONFIG_DISPLAY_WIDTH, CONFIG_DISPLAY_HEIGHT) so it can be easily changed for different e-ink displays.

2.3. Thread Safety and Concurrency

LVGL is not thread-safe. Since the HTTP server runs its API handlers in different FreeRTOS tasks (from the httpd thread pool), we must protect LVGL with a Mutex.

All LVGL initialization, UI setup (lv_obj_create, etc.), and the periodic lv_timer_handler() will take the g_LvglMutex.
The GET /api/display/image API endpoint will acquire g_LvglMutex, draw the screen, encode the BMP header, transmit the buffer, and then release the mutex.

3. Architecture

3.1. File Structure

Provider/main/
├── lv_setup.hpp                 # LVGL initialization and mutex declarations
├── lv_setup.cpp                 # Driver setup, PSRAM buffer allocation, LVGL tick task
├── api/display/
│   ├── image.cpp                # GET /api/display/image handler (BMP streamer)
│   └── unity.cpp                # Aggregator for display endpoints
├── http_server.cpp              # Modified to include api/display/unity.cpp
└── main.cpp                     # Starts LVGL task before HTTP server

3.2. Data Flow

Client makes a GET /api/display/image request.
image.cpp handler intercepts the request and takes the g_LvglMutex.
The handler forces LVGL to finish any pending rendering.
The handler uses lodepng to compress the raw frame buffer into a PNG payload.
The handler sends the PNG via httpd_resp_send().
The handler frees the PNG payload buffer and releases g_LvglMutex.
The HTTP response completes.

4. Design Decisions & Trade-offs

Decision	Trade-off	Rationale
PSRAM Allocation	Slower access than SRAM	A full framebuffer for high-res displays exceeds internal SRAM. PSRAM is required, and since we only read/write it occasionally for HTTP, the speed penalty is negligible compared to network latency.
PNG over BMP	More CPU overhead	PNG compression takes CPU cycles and temporary RAM, but significantly reduces network payload, which is better for web clients and saves transmission time.
Pull vs Push	Requires polling	The client must actively request the image. This is standard for web, but means the client won't know instantly if the UI state changes unless using WebSockets/SSE (future scope).
Synchronous Render	Blocks HTTP task	The HTTP handler waits for LVGL to finish drawing and PNG encoding to complete. Since LVGL renders very quickly in memory, this block should be acceptable.

5. Next Steps

Add lvgl/lvgl to idf_component.yml.
Configure LVGL in sdkconfig to use custom memory allocation (routed to PSRAM).
Implement lv_setup.cpp and api/display/image.cpp.
Draw a sample UI on the virtual display.

Created by Antigravity - Last Updated: 2026-03-14

4.9 KiB Raw Blame History