BoostClock
Frame Time Analysis guide rev 1.0

In this short guide we would like to provide a quick overview of the overlay you can find on our YouTube Frame Time Analysis (FTA) videos.

Layout of the analysis video

We precede every game footage / compute test with a detailed specification of the software in use (build version, graphics options, quality presets, etc.) for the sake of reproducibility and comparability. This is followed by a summary of the hardware configurations that we want to review. The build descriptions are color-coded as the build / hardware legends on the actual footage. The FTA overlay is made up of 2 charts, colored hardware name legends and their respective FPS counters.

The all-important frame times

First of all, to understand and appreciate these charts, you have to realize the importance of measuring the actual frame times. The old school way of measuring performance is done with tools such as FRAPS. The main issue with these kind of methods that they inherently report an averaged value over the course of a second - the number of frames produced by the GPU. To be more precise, FRAPS measures the interval between successive "Present calls" and the current FPS is simply the number of Present calls averaged over 1 second. When it comes to real-time visualization / compute tasks, one second is a very long time and the observer may miss interesting hardware / software characteristics. Furthermore, ranking different hardware with only minimum / average frame-rate (FPS) values really doesn't tell the whole story.

Nevertheless, FPS bar charts are undoubtedly the most convenient form of assessing different hardware setups and of course all the values are still hold true. Enthusiast just needed a new tool to survey how hardware actually performs at any given point.

Graphics anomalies

So what are we after when we deploy frame time analysis? As you may have guessed now, we want to track down graphics anomalies that are perceptible only for a very limited amount, but can severely affect the smoothness of the output image sequence. This is the information that is simply thrown away by the averaging effect.

The most basic anomaly that can occur is stuttering, which is the sudden, abrupt change of pace of the frames presented at the display. It is complex issue that can be rooted in the driver, application and the OS or the interaction of these with each other. No matter where the stuttering happened, the result is an uneven presentation of the frames at the display.

FCAT

Having said that, the most straightforward way of capturing stuttering is at the display level, this is what you experience at the end of the day. Fortunately, NVIDIA provided a simple tool, FCAT that applies a colored border overlay for every frame to the output of the to be reviewed PC build. We record this footage on a separate PC with a capture card in it and analyse the colored sequence. With some simple number crunching, we can easily deduce the frame times for every single frame from the height of the colored bars. As the proper sequence of colors is defined, if we encounter a missing color we can immediately report a dropped frame.

If you want to read more about how the attention turned to stuttering and latency, [1] presents a comprehensive overview on the issue. [2] provides a great summary of the state of game performance testing in 2013. I would recommend [3] as the best self-contained article on the topic where you can read about how all this culminated in FCAT – [4].

Back to the charts!

On our overlay the smaller chart displays the frame times, while the bigger one the frame-rates aka FPS values. The time axis on both charts spans through 6 seconds with current time at the center. We may occasionally add a "slider bar" which shows the performance difference between the reviewed setups.

Review of the results

To round off the analysis video, we repeat the software settings used for benchmarking and display the most important aspects of the builds as well. And what's more, we provide a frame time percentile graph [5] and report maximum, average, minimum and 97% percentile FPS values, all drawn from the FCAT runs. We concluded the review by the FPS and frame time charts in their full glory.

We are hopeful that our readers / viewers will find our frame time benchmarking interesting and regularly come back for more deep dives into games and hardware.