This article explains the mechanics behind TFT controllers and their central role in relation to the timing and performance of TouchGFX.
The TFT controller on an MCU is responsible for updating the display glass with the pixel values found in the frame buffer. The display glass itself does not have any memory, so the pixel values must be transmitted to the display continuously, regardless of whether or not the frame buffer contents change. So even when displaying a still image, the frame buffer is being transferred to the display.
The display update frequency varies among displays, but it is usually in the 50-60 Hz range. That means, every 16-20 milliseconds the frame buffer contents are being transferred. The transfer usually happen in bursts with a shorter duration than the update frequency, leaving some idle time between each update where the memory holding the frame buffer can be accessed for other means (like, for instance, writing new pixel values in the frame buffer).
The following figure shows the update cycle of a TFT controller on a 320x240 pixel display:
The porch areas are delays where no data is being sent. Pixel data is transferred one pixel at a time, starting from the top left corner of the display (pixel 0,0) traversing horizontally, with delays before and after each line (horizontal porches) and delays before first line and after last line (vertical porches). For every display update (frame) a VSYNC signal is issued, and for each line in that frame, a HSYNC signal is issued. Whenever the controller is inside the active area, the SDRAM containing the frame buffer is busy. In order to optimize SDRAM access, the controller signals are directly connected to when TouchGFX performs its rendering, as explained in the following.
A frame rendering pass begins when the TFT controller enters the active area. The rendering pass consists of the following steps:
- Call the virtual beginFrame function
Application::handleTickEventfunction, which will tick timer widgets and the active view. This step may cause invalidation of the screen. Invalidation means that the contents of the display glass and framebuffer do not reflect the current layout of the screen, so the framebuffer needs to be redrawn to make the display glass show the correct information on the screen.
- Sample touch and forward touch events to the application. This step may also cause invalidation.
- Sample keys and buttons if appropriate and forward these to the application. This step may also cause invalidation.
- Coalesce invalidated areas and begin actual drawing where needed
- Wait for drawing to be completed
- Call the virtual endFrame function
If the time it takes to render the screen is less than the display update rate, the actual frame rate of the system will match the display update rate. This will usually happen if nothing or only a small portion of the screen needs to be drawn. If, however, the drawing takes longer, the actual frame rate will decrease because the next frame rendering pass will not happen the next time the TFT controller enters the active area, since the previous drawing was still underway. The following figure outlines the timing associated with a typical rendering pass, spanning four frames, where all rendering passes take less time than the display update frequency.
What you see above is that the GUI task begins its frame rendering immediately when the TFT controller enters the active area (ie. begins updating the display), labeled a). The GUI task will sample touch and buttons, and call tick on the active view and any widgets that have registered for timer events. In this example, this causes an invalidation of a region of the screen. At b), rendering is completed, and the GUI task can now sleep. At c), the TFT controller begins updating the display (entering the active area for the next frame). Here several things happen:
- Since we have redrawn parts of the screen, we swap frame buffers, such that the TFT controller will use the frame buffer in which we just drew as base for updating the display in this cycle.
- Additionally, the GUI task wakes up again and begins the next frame rendering pass. In this example figure, no screen changes were required in this pass, so the GUI task will quickly sleep (d) and the system idles until next frame.
- You will also note that no frame buffer swap takes place at next display update (e) because nothing was drawn.
If the screen area that needs to be drawn is large, it is quite possible that the drawing operations take longer than the display update frequency, as illustrated here:
Here you will notice the rendering process is still underway when the next TFT controller update begins (a). Since the previous frame was not complete, this event is ignored so no new frame rendering pass begins here. The frame buffer swap will occur at next TFT update after the rendering is completed (b). In this example, the effective frame rate (ie. the frequency with which the pixels on the display actually change) is VSYNC/2. If the drawing operation took a very long time, it might even span across three or four updates. This is not a problem if it only happens on occasions such as switching to a different view. But if it happens during an animation that is supposed to look fluent the effective frame rate will be too slow.
Because the frame buffer swapping takes places at the next display update after drawing is complete, it may be necessary to adjust the VSYNC frequency in order to obtain the optimal timing. Consider the following example:
What you see above is that rendering completes immediately after the display begins updating (a). At this point the frame buffer swap cannot occur because the TFT controller is in the middle of updating the display. Therefore the swap cannot take place until next update, which means that the pixel update will not be visible until next update (b). This yields an effective frame rate of VSYNC/3. In such a scenario it would actually be better to slow down the TFT update frequency e.g. by extending the vertical porches. The following figure illustrates the same drawing operation, but with a slightly lower VSYNC frequency:
The porch delay area has been slightly extended (a). Now the rendering completes immediately before a TFT update, thus allowing the frame buffers to be swapped earlier, yielding a better effective frame rate.
Because of this, the optimal VSYNC timing of your system can be hard to determine before the UI has been implemented. A good rule of thumb is to aim for a 16-20ms update frequency initially, and investigate the actual rendering times of your application to fine tune.
For the sake of simplicity, all the examples above simply show the rendering time and does not distinguish whether the rendering is done by DMA or MCU. TouchGFX will attempt to use the DMA as much as possible in order for the MCU load to remain low. Depending on your target microcontroller and the composition of your user interface, it is not uncommon for the GUI task itself to also perform rendering, however, for instance when drawing text or alpha blended images if this is not supported by the DMA engine. See Blitting Operations and Hardware Blit Capabilities for details on hardware support. If software rendering is involved, a more detailed timing diagram is as follows:
In the above figure we see the GUI task wake up (a) and begin processing the frame (sampling touch, handling timer ticks, create a queue of DMA operations). When this is done, the DMA takes over (b). We then see the GUI task performing MCU based rendering at c), and with subsequent interleaved DMA and MCU based drawing for the remainder of the rendering process. The synchronization of the interleaving process is built into TouchGFX and controlled by a frame buffer semaphore, which allows the GUI task and the DMA to alternately perform drawing operations in the frame buffer.
Compensating for reduced frame rate
The tick methods are called once for each call of draw. That is, one tick for each new frame. In case of having frames that occasionally take a long time to render, the tick frequency may be irregular.
The periodic tick can be used to implement animations: As an example an icon should move 200 pixels across the screen in ~3 seconds. If the LCD update rate is 60 Hz (corresponding to 16.7 ms) the icon needs to move 1 pixel in every frame.
This will work nicely if the drawing operations can complete within the 16.7 ms. If the drawing operation takes longer than that, the next LCD update is missed and the frame rate is reduced. Usually this is not a big problem, as the animation would be modify to, e.g. move the icon 2 pixels in every frame. In this case, the animation would still complete in 3 seconds. But it will be problematic in cases where the tick rate is irregular - i.e. sometimes the tick rate is 16.7 ms and sometimes it is 33 ms as it would cause the animation to look uneven, and the duration would be unpredictable.
To alleviate this, TouchGFX has a "reduced frame rate" compensating feature that relieves this problem. When enabled it will make an additional call to tick for each missed LCD update and the rate of calls to tick will thus match the LCD update rate.
In the above example the tick method will be called 3 times prior to calling draw after the long running draw operation. It is now sufficient to move the icon 1 pixel in tick.
Frame rate compensation is as default disabled.
It is enable by calling the setFrameRateCompensation method in HAL:
If this is needed as a general setting for the application it is recommended to insert the call in touchgfx_init() in BoardConfiguration.cpp. Otherwise it can be enabled/disabled on a per-View basis.
TouchGFX can provide the number of frames missed per tick by calling the getLCDRefreshCount method in HAL:
This can be used to skip time consuming actions in the tick method by checking the size of the LCDRefreshCount. It will be equal to the number of draws that has been executed since the last tick, which, in the case of FrameRateCompensation, will be equal to the number of ticks that will be executed in this frame.