So you want to include images into your LVGL project, running on a nRF5340. You prepare your images, convert them using LVGL online tool, integrate them into your application, and.... end up with 167% of your flash used!
Optimizing your images is crucial in low resources system such as MCUs. As a quick example, if you want to use a full size background for your application, using a 480x320 display, using ARGB8888 you end up with:
480 * 320 * 4 = 614400 B = 614.4 kB
Which, in some system corresponds to more than half the flash size.
So what can we do to optimize it? The following sections will show solutions to reduce flash usage.
Using RLE compression
RLE is a simple compression format which group contiguous symbols together.
Here is a simple example:
AAABBCCCCD (10B) -> RLE compression -> 3A2B4C1D (8B)
It is very useful when your image have a lot of contiguous same color pixels. It becomes a bit less interesting with images using a lot of colors and blurring as there won't be that much contiguous pixels.
To convert an image to RLE, LVGL offers an utility script in scripts/LVGLImage.py:
python3 lvgl/scripts/LVGLImage.py --ofmt C --cf RGB565 --compress RLE -o my_image.c my_image.png
This will convert my_image.png to a .c file in RGB565 format, compressed using RLE. To be able to use this compressed image in LVGL, you must activate these configs:
- LV_USE_RLE 1: Enable RLE processing.
- LV_BIN_DECODER_RAM_LOAD 1: Enable decoding the binary images in RAM.
Note that RLE decompression can use quite a lot of RAM, as it will need decompression buffers while loading the image.
Using indexed color format
The indexed color format allow to represent each pixel as an index, pointing to a color in a palette,which is encoded next to the pixels data. It means that if your image has <= 256 colors, the palette index can be represented as 1B (index 0 to 255). The palettes are written as ARGB8888, meaning that a 256 colors palette = 256 * 4 = 1024B = 1kB.
To convert an image to indexed color, you can also use LVGL scripts/LVGLImage.py:
python3 lvgl/LVGLImage.py --ofmt C --cf I8 -o my_image.c my_image.png
The script internally convert your base PNG to an indexed PNG if needed, and then convert it to a .c.
One can use these color format (parameter --cf):
- I1: 1 color palette
- I2: 4 color palette
- I4: 16 colors palette
- I8: 256 colors palette
Using A* format (for grayscale images)
The last format(s) we will talk about are the A* formats, where * can be 1, 2, 4, 8. This format allows to only encode the transparency. Like this, you end up with a grayscale image, which can the be recolored using the style.recolor property.
To convert an image to A8, you can also use LVGL scripts/LVGLImage.py:
python3 lvgl/LVGLImage.py --ofmt C --cf A8 -o my_image.c my_image.png
Conclusion
We saw that multiple options are available when you need optimizing your images size. Depending on your base image, it may be better to use one or the other method, so my advice would be to try playing a bit with these color formats and see what works best for your images.
If you need any assistance in optimizing your images for your project, don't hesitate to contact us!
Links:
- RLE compression: https://en.wikipedia.org/wiki/Run-length_encoding
- Indexed color format: https://en.wikipedia.org/wiki/Indexed_color#:~:text=In%20computing%2C%20indexed%20color%20is,2%2Dbit%20indexed%20color%20image
- LVGL Image doc: https://docs.lvgl.io/9.2/overview/image.html#overview-image
If First of all... what is it? To keep it simple, it is a multi-layered system that manages rendering and display output.
In this post, we'll explore the Linux graphics stack, focusing on key components involved in integrating DRM driver support into LVGL.
Overview
The following diagram shows all the core elements/libraries to display something on a screen from an application.
There are many libraries and concepts involved to get something displayed on the screen. Let's dive into all these elements!
Some context
There are several other ways to display graphics on a Linux system:
- Framebuffer (uses /fbev/fb0). Simple, works on almost any hardware, but has no hardware acceleration and is outdated.
- Wayland. Window manager and compositor (can use DRM).
- SDL (Simple DirectMedia Layer). Handles graphics, input and audio.
- DRM/GBM/OpenGL. Go to the next sections for details!
What is the DRM (Direct Rendering Manager)
The DRM is a Linux kernel subsystem that manages graphics hardware. It provides a unified interface for handling GPU acceleration, framebuffer management, and display output. It ensures efficient rendering while allowing user-space applications to communicate with the graphic hardware.
DRM uses KMS to configure the display settings and set the framebuffer, connectors, and display pipelines to drive the actual screen output.
So... about GBM?
GBM (Generic Buffer Management) is a Linux API that allows applications to allocate GPU buffers without relying on a windowing system like X11 or Wayland. It is primarily used in DRM/KMS-based rendering pipelines to manage graphics memory efficiently.
Ok, and EGL/OpenGL?
OpenGL is the rendering API. It is a cross-platform API used for rendering 2D and 3D graphics. It provides a set of functions that let applications send drawing commands to the GPU. Great! But how?
EGL! is an interface that connects OpenGL with the native windowing system or display hardware (KMS/GBM). EGL creates the graphic context in within OpenGL can render. You can see EGL as the glue between GBM buffers and OpenGL rendering.
OpenGL ES
The name itself gives a hint: Embedded System. It's a lightweight and optimized version of OpenGL that is designed for mobile and embedded devices.
GPU drivers
We will not cover anything about the drivers here, we assume the drivers exist and can be used with the GPU of the system.
Details about the interactions
To make it simple:
- DRM library is used to handle the display settings
- GBM is used to create the buffers.
- EGL uses GBM buffers to create EGL context, creating EGL buffers from GBM.
- OpenGL ES uses EGL context to render elements into EGL buffers.
Instead of OpenGL ES, it's possible to use OpenGL or Vulkan. EGL creates a graphic context that can be used by any rendering framework.
DRM in LVGL
LVGL had a DRM implementation using "dumb buffers" to display a buffer on the screen. They are designed to be easy to use but lack GPU acceleration, making them inefficient for advanced rendering tasks. It means it relies on CPU software rendering, being very similar to framebuffer (fbdev).
Why do dumb buffer exist?
Well, they are usually used for debugging and testing low-level DRM/KMS functionality. They are also used as a fallback mode when no GPU acceleration is available. But our objective here is to use the GPU to be more efficient, so we modified the existing DRM support to use:
- GBM as buffer allocator
- EGL to provide a graphic context
- OpenGL ES to render in the context
This feature is coming soon! LVGL v9.3 will have the DRM support using hardware acceleration! Stay tuned 😉
