Why Texture Compression?

Given that computer memories have gotten bigger, faster, and cheaper over the years, you might wonder whether there is still a need for textures to be compressed in memory at all. (Compression on disk is another matter, but I’m not talking about that in this article.) But at least for large-scale, realistic 3D games, aesthetic expectations have of course grown in step with the hardware improvements. There’s a continuing push for more textures with higher resolutions, to increase visual detail and reduce repetition; and as game shading models become more sophisticated, each material requires a greater number of textures: diffuse color, normal map, specular color, gloss, emissive glow, and others.

Moreover, despite the vast advancements in hardware capabilities over the last couple of decades, texture sampling in a shader is becoming relatively more expensive, not less! Processors have gotten faster and faster over the years, and so has memory—but memory is falling behind, and the gap between memory and computation has grown wider. Hardware designers are fighting an uphill battle to provide enough memory bandwidth to keep processors supplied with data to chew on. Given this state of affairs, texture compression is not just about cramming more pixels into the game; it’s a crucial performance optimization, since it reduces by a large factor the memory bandwidth required to get those pixels into the GPU’s shader cores.

It’s clear that GPU-supported texture compression is not going to go away. In fact, it is more vital than ever to building a game with high-quality graphics—and new, more sophisticated compression formats are a continuing area of research.

Block Compression

BC stands for “block compression”, and the BCn formats all operate in terms of 4×4 blocks of pixels. All images are sliced up into these small blocks, and each block is self-contained—the data to decode it is all in one contiguous chunk in memory. Moreover, the size of each compressed block is fixed—either 8 or 16 bytes, depending on which BCn format is being used. This represents a 4:1 or 8:1 compression ratio, if the source image is in 8-bit RGBA format.

This standard layout is designed to make it easy for the GPU to use these formats for rendering. GPUs need to be able to quickly access any part of a texture; they don’t read the entire image sequentially, so streaming compression algorithms and those that have variable compression ratios are poorly suited for this application. The fixed block size makes it easy to locate the BCn block containing any given pixel, and the self-contained block data means that the GPU can decompress just the part of the image it needs to access.

Moreover, texture samples often exhibit locality in both dimensions—if one pixel from a texture is accessed, it’s likely that other pixels nearby in 2D will be accessed too. The 4×4 block structure supports this well, since it’s convenient to decompress all 16 pixels at once and then store them in the texture cache, where they can be efficiently reused for additional sampling operations.

The first three BCn formats are better known as DXT1, DXT3, and DXT5, respectively, but BCn (for n between 1 and 7) are the more up-to-date names for these formats.

Endpoints And Indices

All of the BCn formats are based around a simple idea: in many images of interest, within any small area (such as a 4×4 block) there tends to be limited variation in the set of colors present. For example, take a look at this brick texture (from CgTextures). An extract from the texture is shown here, together with several magnified 4×4 blocks from it:

If you look at these blocks, you can see what’s meant by “limited color variation”. Within a block, we often have just darker and lighter shades of a single color, or (at corners and edges) a gradient or blend between two contrasting colors. The BCn formats are designed to exploit this redundancy by separating the definition of the colors in a block from their spatial distribution.

Each block in a BCn image has a very small color palette, with just a few colors to choose from. The pixels are then coded as indices into this palette, which requires only a handful of bits per pixel, since the palette is so small. The palette is further compressed by assuming that all its colors are laid out evenly spaced along a line segment in RGB space. Then the file only has to store the endpoints of that line, and all the other palette entries are reconstructed by blending the two endpoints in different proportions.

All BCn blocks contain two main pieces of data: the endpoints of these color-space line segments, and the per-pixel palette indices that say how far along the line each pixel is.

Even without delving into the details of specific BCn formats, we can already get an idea of the kinds of situations in which they will fail. Most of the BCn formats will give poor-quality results anywhere that three very different colors are present in a single block. For example, a block containing a mix of red, green and blue pixels cannot be represented using the simpler BCn formats, because red, green, and blue do not lie along a straight line in RGB space. This presents a problem for normal maps, which often have exactly this situation. On the other hand, color textures like the brick shown above can often survive BCn compression with very little visible degradation.

As an example, here are the four blocks from above, compressed using BC1 format:

Here you can clearly see how each block has collapsed down to no more than four distinct colors (BC1 uses a four-color palette), and subtle hue variations have vanished (because the four colors must lie on a line in RGB space). The effects are obvious at the level of individual blocks, but at the level of the entire image they are much less apparent:

If you look carefully, you can see definite differences between these two images—particularly along the edges between shadowed and bright areas—but on the whole, the compressed image looks very faithful to the original.

Now that we’ve gone over the basics, let’s jump into the details of specific BCn formats.

BC1

BC1 stores RGB data. It technically supports an alpha channel, but the alpha is only 1-bit (that is, it must be either 0 or 255). It uses 8 bytes to store each 4×4 block, giving it an average data rate of 0.5 bytes per pixel. Each block consists of two color endpoints, which are stored in 2 bytes each, using RGB 5:6:5 format. The palette contains four entries generated from those endpoints, so the indices require 2 bits per pixel, making up the other 4 bytes of the block.

BC1 is a good choice for most standard-issue color maps, unless there’s a specific reason to use one of the other formats. One such reason could be that the image requires smooth gradients. Due to the use of 5:6:5 colors, BC1 cannot represent smooth gradients well, as illustrated here:

The top gradient should look relatively smooth (although on many monitors you will still see some banding—with today’s contrast ratios eight bits aren’t enough for perfectly smooth gradients). However, the bottom gradient is much more bandy than the top one. Some bands are even visibly green, since the 5:6:5 encoding gives us colors like (57, 60, 57).

This isn’t an issue with fitting pixels into a four-color palette. The gradients above are 512 pixels wide, which means that each 4×4 block contains exactly two colors—no issue for a four-color palette to handle! The problem is that the endpoints aren’t stored with enough precision. However, this is only an issue for certain kinds of textures that involve very smooth gradients or very subtle color variations, such as skies and human skin. For these kinds of images, BC7 (described later) may be a better choice. The majority of game textures undergo little visible degradation in BC1, like the brick texture in the previous section.

Degeneracy, and Breaking It

I mentioned earlier that BC1 supports a 1-bit alpha channel. How does this fit in? After the 4 bytes of endpoints and 4 bytes of indices described above, there’s apparently no more space in the block for additional data. But there is a loophole that can be exploited to pack even more information into the same space: degeneracy! The system described above is degenerate, meaning that there are multiple ways to encode the same image. In this case, the degeneracy originates in the symmetry of the two color endpoints. Nothing about the BC1 format thus far singles out in what order the endpoints should be stored: if you swap the two, and invert the indices to compensate, you end up with exactly the same 4×4 block of pixels as before. So there is a twofold degeneracy: there are two equally good ways to store the same image.

BC1 cleverly exploits this by breaking the degeneracy: it defines an alternative mode that is triggered for a given block by the order of the endpoints. Although the endpoints are colors in 5:6:5 format, they can also be interpreted as 16-bit unsigned integers. If the first endpoint is numerically greater than the second, the above description of the format holds: the palette contains four colors spaced evenly from one endpoint to the other. But if the first endpoint is less-equal to the second, the palette is modified: its first three entries are three colors spaced evenly from one endpoint to the other (that is, the two endpoints and their average), and the fourth entry is transparent: black with zero alpha.

In this way, BC1 can support a 1-bit alpha by switching into this second mode for specific blocks that contain transparent pixels. However, some precision must be sacrificed for the non-transparent values in these blocks because only three distinct colors remain, rather than four. BC1 also cannot store any color information in the transparent areas. This makes it suitable for storing texture maps for “cutout” materials, such as grates, fences, and vegetation, where alpha testing is used to discard the transparent parts of the image. Care must be taken, however, when using bilinear filtering with BC1 cutout textures. Since the color component of the transparent pixels is always black, a dark fringe will form around the transparent areas where bilinear filtering blends the colored pixels with the transparent ones. This can be avoided by setting the alpha threshold high, so that the dark areas are culled away, or by dividing the interpolated color by the interpolated alpha in the shader, which will cancel out the darkening.

Here is an example of an image with 1-bit alpha, upscaled with bilinear filtering, with an alpha test threshold of 128. On the left, only plain bilinear filtering has been used, producing a dark fringe where the color of the texture is interpolated 50% toward the black of the transparent area. On the right, the interpolated alpha has been divided out.

BC4

Rather than going through these formats in numerical order (which corresponds to the chronological order in which they were introduced, in successive generations of GPU hardware), I’m going to go in order from the simplest to most complicated. After BC1, BC4 is the next logical step.

BC4 stores a grayscale image—no RGB, just a single color channel—and uses 8 bytes per block. Its endpoints are one byte each, and it uses an eight-element palette, so it has 3 bits of indices per pixel.

BC4 is the same size as BC1, but it gives much better quality than BC1 when storing a grayscale image, due to both the expanded palette (eight elements instead of four) and the extended endpoint precision (8 bits instead of 5–6). This makes BC4 an excellent choice for height maps, gloss maps, and any other kind of grayscale texture. Compare the quality of the gradient here with that of BC1, above:

There is little or no visible difference between the BC4-compressed gradient and the uncompressed original.

Like BC1, BC4 makes use of degeneracy breaking by defining an alternative mode triggered based on the order of the endpoints. If the first endpoint is numerically greater than the second, the palette consists of eight values evenly spaced from one endpoint to the other. Otherwise, the first six entries in the palette are evenly spaced from one endpoint to the other, and the last two are, respectively, 0 and 255—black and white. BC4 already has excellent quality due to its full 8-bit endpoints and large palette, but allowing this alternative mode can make it even better in certain cases, such as at sharp edges between black and white areas in the map.

BC2, BC3, and BC5

These formats are simply combinations of the previous two. BC3 stores RGBA data, using BC1 for the RGB part and BC4 for the alpha part, for a total block size of 16 bytes, or an average of 1 byte per pixel. It’s the most common format for textures that require a full alpha channel, and can also be used for packing a color texture together with any grayscale image, such as a height map or gloss map. Since the alpha is stored separately from the color, BC3 does not use the BC1 1-bit alpha mode in the color part.

BC5 is a two-channel format in which each block is just two BC4 blocks. This is very useful for tangent-space normal maps, if the the X and Y components are stored and the Z component is reconstructed in the pixel shader. Since each channel has its own endpoints and indices, normal maps—in which the X and Y components are often “doing different things”, so to speak—retain quite a bit more fidelity in BC5 than in BC1. The downside is that BC5 requires twice as much memory, at 16 bytes per block; this can also make it slower for shaders to access because more memory bandwidth is needed to get the texture to the shader cores. But this may be a price worth paying for the substantial increase in quality.

BC2 is a bit of an odd duck, and frankly is never used nowadays. It stores RGBA data, using BC1 for the RGB part, and a straight 4 bits per pixel for the alpha channel. The alpha part doesn’t use any endpoints-and-indices scheme, just stores explicit pixel values. But since each alpha value is just 4 bits, there are only 16 distinct levels of alpha, which causes extreme banding and makes it impossible to represent a smooth gradient or edge even approximately. Like BC3, it totals 16 bytes per block. As far as I can think of, there’s no reason ever to use this format, since BC3 can do a better job in the same amount of memory. I include it here just for historical reasons.

BC6 and BC7

The final two formats were introduced very recently, just within the last couple of years, and are only supported by D3D11-level graphics hardware. They’re also vastly more complex than any of the other formats we’ve discussed. As a result of their newness, complexity, and hardware requirements, they’re not yet well supported in texture compression tools and libraries, and aren’t yet well-known or widely used.

Both of them consume 16 bytes per block, the same as BC3 and BC5. BC7 targets 8-bit RGB or RGBA data, and BC6 targets RGB half-precision floating-point data. BC6 is therefore the only BCn format that can natively store HDR images, and is an excellent replacement for RGBM and other HDR encodings that rendering programmers have heretofore used to shoehorn HDR data into compressed textures.

The reason BC6 and BC7 are so complicated is that they allow a variety of different modes that change the details of the format, such as the palette size and the way the endpoints are stored. Modes are specified by the first few bits of each block, so each block can effectively have a different format! This makes BC6 and BC7 very adaptable to the image contents, as they can choose the best mode for each individual 4×4 block. But the downside is that compressing to BC6 or BC7 is much more difficult and slow, since the compressor has many more options to try to achieve the best-quality representation of each block.

The different modes essentially trade off various features of the format. For example, they trade off endpoint precision versus index precision: some modes have larger palettes, but store the endpoints with fewer bits per component; other modes have higher-precision endpoints, but smaller palettes.

Another enhancement that BC6 and BC7 feature is the ability to have more than one line segment in each block. Now, some of the formats described previously—namely BC3 and BC5—have two line segments per block, but they use them for different channels—in BC3, color and alpha, or in BC5, the two grayscale channels. BC6 and BC7 introduce the concept of partitioning, which allows different line segments to be used for different pixels in the block. There are a variety of spatial partitioning patterns available (the ones for BC6 can be seen here). These predefined patterns assign each pixel in the block to one line segment or another; the indices then control which color out of that line segment’s palette the pixel gets. Partitioning can improve quality in cases where the colors in a block don’t fall very neatly along a single line in RGB space; with multiple line segments, the original range of colors in the block can be reproduced more faithfully.

The number of line segments is controlled by the per-block mode setting. Modes with more line segments have more endpoints to store, so naturally, these modes also tend to have lower-precision endpoints and smaller palettes—everything still has to fit in 16 bytes per block. When applicable, the chosen partition pattern is also stored by a few more bits in the block.

Image Comparisons

As an example of the higher image quality enabled by the sophisticated BC7 features, here are the four blocks extracted from the brick texture earlier in this article, compressed with BC7:

If you compare this with the earlier BC1 version of this image, there’s a massive difference. The second block from the left still appears visibly degraded relative to the uncompressed version, but it’s much better than in BC1, and the other three blocks hardly have any visible differences with their uncompressed versions.

At the zoomed-out level, there is no perceptible difference between the uncompressed and BC7 versions of the image, even if you examine them quite closely:

Finally, BC7 also works extremely well on the gradient example, where BC1 failed miserably:

Compression Cleverness

The slightly unfortunate thing about BC6 and BC7 is that because they have all these different per-block options, they have to use up some of the precious 16 bytes just to say which options were picked, leaving less space for the actual contents—the endpoints and indices. However, the smart people who invented these formats (presumably some engineers at NVIDIA and AMD…will we ever know who they are?) found a whole bag of tricks for squeezing more data into a block.

For example, recall that BC1 and BC3 both exploit degeneracy breaking to effectively gain one more bit of data, by using the order of the two endpoints in a block as a signal to switch between two modes. BC6 and BC7 also use degeneracy breaking, but not to switch modes; they take advantage of it to eliminate one bit of the indices. When you swap the two endpoints, you have to bitwise-invert all the indices to compensate: indices 00, 01, 10, and 11 become 11, 10, 01, and 00, respectively. But you can also turn this around: you can always invert the indices by swapping the endpoints. So you can declare, for instance, that the most-significant bit of the upper-left pixel’s index will always be 0: if it’s not, swap the endpoints and then it will be! Then you don’t have to actually store the 0 bit; that bit can be used for something else. In partitioned modes, one bit can be saved for each line segment by swapping that segment’s endpoints.

Another space-saving trick, which BC6 uses in most of its modes, is delta compression for endpoints. Rather than storing all the endpoints at a high level of precision, these modes store one endpoint at relatively high precision, then represent the other endpoints as lower-precision delta vectors relative to that base endpoint. (Unfortunately, the spec refers to this feature by the uninformative phrase “transformed endpoints”.) This is an interesting approach because it restricts the possible lengths and orientations of the line segments in RGB space, but still allows their absolute position to be set precisely. Quantized lengths and orientations for the line segments will naturally introduce error in color reproduction, but precise absolute positioning allows the error to be distributed more evenly over all the pixels, as opposed to concentrating in a few.

Incidentally, although BC6 deals in floating-point values, it nevertheless treats them as 16-bit integers throughout almost all stages of the decoding and interpolation process! If you’re wondering how that can even work at all, take a look at this article. The key point is that magnitude order—from lower to higher values—is the same for floats and their integer representations. So interpolating floats as if they were ints actually does something reasonable, although it does not always produce linear interpolation—it effectively interpolates along a piecewise-linear approximation of a logarithmic curve! BC6 does involve some special-casing to handle negative numbers, NaNs and infinities, but it mostly treats its values as ints, relying on this fact about the IEEE float representation to make everything work.

BC7 does not use delta compression for its endpoints, but has a similar mechanism, referred to as “P-bits”. A P-bit is a shared least-significant bit that gets tacked onto the end of all the RGB color values of the endpoints. This is a bit of a head-scratcher, but the end result of it is very like that of BC6′s delta compression: the possible lengths and orientations of the RGB line segments are more coarsely quantized, but the whole line segment can be positioned more precisely, allowing error to be distributed more evenly over the pixels in the block.

Finally, since BC7 can store both color and alpha channels, it’s equipped with modes that offer a few choices for combining the two. Some modes have one set of indices for all four channels, which (roughly speaking) requires color and alpha to have the same spatial distribution within the block. Other modes include two distinct sets of indices, one for color and one for alpha, allowing the two to be relatively independent. Moreover, the separate-index modes also include channel swapping flags: alpha can be swapped with red, green, or blue, or left in place. This effectively allows any of the four channels to use distinct indices from the rest.

Comparison Table

To sum up all up, here’s a table listing the major differences between the seven BCn formats:

Compressors

In this article, I described the data representation of the BCn formats, but not how to write compressors for them. Like many compression techniques, the BCn formats are designed to be simple and fast to decompress—but that often comes at the cost of making compression difficult! Writing high-quality BCn compressors is a big enough topic for its own article, and is a subject of ongoing research. In particular, the BC6 and BC7 formats have a much greater “search space” because they offer so many more options for encoding each block of an image, and high-quality BC6 and BC7 compression involves a lot of brute-force searching for the best (lowest-error) combination of options, making these compressors quite a bit slower than those for BC1–5.

For compressing and decompressing BC1–5, the open-source NVIDIA Texture Tools are probably your best bet. NVTT includes both command-line utilities and a set of libraries that can be linked into your own projects.

It is still surprisingly difficult to find any publicly available tools that support BC6–7 at all. The only publicly available tools I’ve found that can compress to these formats with high image quality are NVIDIA’s internal development compressors, which have been open-sourced and are hosted at the NVTT Google Code repository here. These tools are not the user-friendliest! They run very slowly—taking several minutes to encode a 256×256 image—and they don’t save to a standard file format like DDS, but simply dump the compressed blocks into a raw binary file. Their license status is also unknown, so it may not be safe from a legal perspective to incorporate their source into your own projects. However, they do give very high-quality results, and I used them for all the BC7 comparison images in this article.

Compression and decompression of all the BCn formats is also supported by the Direct3D 11 SDK in the form of the D3DX11CreateTextureFromFile function. However, my experience is that the compressed image quality is not very good with this API, so I would not advise using it for compression (decompression should be fine).

References

For the full, bit-for-bit specifications of each of the BCn formats, see these references:

• Overview of all the BCn formats
• Complete specs for BC1–5
• Complete specs for BC6–7

In textbooks and university computer-science courses, we often hear about a few classic approaches to parsing formulas. One is the so-called reverse Polish notation, where we write formulas in postfix form, with operators following their operands:

2 3 +           // means 2 + 3
a b + c d + *   // means (a + b) * (c + d)
pi 4 / sin      // means sin(pi/4)

The nice things about RPN are that (a) parentheses are not needed, nor the concepts of operator precedence and associativity, since the order of operations is fully specified by the notation; and (b) it can be parsed by a simple algorithm: scan the formula left-to-right, when you see an operand push it on a stack, and when you see an operator, pop the required operand(s) off the stack, apply the operator, and push the result back on. When you’re done, the result is the only item left on the stack (assuming well-formed input).

That’s quite easy to add to an application; there’s almost no infrastructure needed. But it has the disadvantage that it requires users to work in this unfamiliar and awkward notation. Of course, users can be trained to work in RPN, and with experience it no longer appears unfamiliar or awkward. But let’s take pity on the poor users and let them work with standard mathematical notation. What options are there?

The standard computer-science cirriculum at this point would start talking about context-free grammars, abstract syntax trees, and syntax-directed parsers. There are two main approaches to building parsers that are used in practice, i.e. for parsing programming languages: top-down (also known as recursive descent or LL) and bottom-up (aka shift-reduce, LR). Unfortunately, neither of these is a good fit for embedding a simple arithmetic language in an application. Top-down parsing isn’t a good fit for arithmetic in general, since each level of operator precedence requires its own nonterminal symbol in the grammar (each of which corresponds to a function call in the parser), and right-associative operators can’t be expressed in the grammar without breaking the LL constraint, necessitating some sort of extragrammatical fixup. Bottom-up parsing works by using a large state machine whose transition rules are usually impractical to work out by hand, requiring a parser generator tool such as Bison to compute. Again, that’s a lot of infrastructure to throw at what is not such a complicated problem.

Fortunately, there is another way: the shunting-yard algorithm. It is due to Edsger Dijkstra, and so named because it supposedly resembles the way trains are assembled and disassembled in a railyard. This algorithm processes infix notation efficiently, supports precedence and associativity well, and can be easily hand-coded.

How It Works

As in RPN, we scan the formula from left to right, processing each operand and operator in order. However, we now have two stacks: one for operands and another for operators. Then, we proceed as follows:

• If we see an operand, push it on the operand stack.
• If we see an operator:
  • While there’s an operator on top of the operator stack of precedence higher than or equal to that of the operator we’re currently processing, pop it off and apply it. (That is, pop the required operand(s) off the stack, apply the operator to them, and push the result back on the operand stack.)
  • Then, push the current operator on the operator stack.
• When we get to the end of the formula, apply any operators remaining on the stack, from the top down. Then the result is the only item left on the operand stack (assuming well-formed input).

Note that “applying” an operator can mean a couple of different things in this context. You could actually execute the operators, in which case the operands would be numerical values of all the terms and subexpressions; you could also build a syntax tree, in which case the operands would be subtrees. The algorithm works the same way in either case.

That’s basically all there is to it, aside from some bells and whistles! As you can see, it has a lot in common with the RPN algorithm, and is just a little more complicated.

Advanced Usage

I described the algorithm above in its simplest form, but there are several enhancements that can be made to handle more complicated formulas.

Associativity. Above, I said that when processing an operator, any operators of equal precedence at the top of the stack should be popped and applied. This makes those operators left-associative, since the leftmost of the two operators will be applied first. You can implement right-associativity by leaving equal-precedence operators on the stack.

Parentheses. Parens are a bit of a special case. When you see a left paren, push it on the operator stack; no other operators can pop a paren (so it’s as if it has the lowest precedence). Then when you see a right paren, pop-and-apply any operators on the stack until you get back to a left paren, which is popped and discarded.

Unary operators. These generally work just like any binary operators except that they only pop one operand when they’re applied. There is one extra rule that needs to be followed, though: when processing a unary operator, it’s only allowed to pop-and-apply other unary operators—never any binary ones, regardless of precedence. This rule is to ensure that formulas like a ^ -b are handled correctly, where ^ (exponentiation) has a higher precedence than - (negation). (In a ^ -b there’s only one correct parse, but in -a^b you want to apply the ^ first.)

Both prefix and postfix unary operators can be used. The way to tell whether you’re in a position to allow prefix or postfix operators is to look at the previous token; if it’s an operand, you’re looking for binary and postfix unary operators, and if the previous token is an operator (or there’s no previous token) you’re looking for prefix unary operators. Note that a left parent counts as an operator and a right paren as an operand for this purpose. This rule also allows you to tell whether - is a negation (unary) or a subtraction (binary)—it’s a negation if it appears when looking for a prefix unary operator, and a subtraction otherwise.

Function calls. The prefix/postfix rule also allows you to tell when an open paren designates a function call rather than grouping a subexpression (a grouping paren is like a prefix operator while a function-call one is like a postfix operator). When a function-call paren is encountered, the operand at the top of the stack is the function to be called. Push the paren on the operator stack as before, but also set up a list somewhere to hold the function arguments, and maintain a mapping that lets you find that list again from the paren on the stack. (Note that with nested function calls, you could have multiple left parens on the stack.)

Then, when a comma is encountered, pop-and-apply operators back to a left paren; the operand on the top of the stack is then the next argument, and should be popped and added to the argument list. When the right paren is encountered, do the same, then pop and discard the left paren. (Note that the arguments shouldn’t be left on the operand stack, at least not without some sort of sentinel between them; this would allow an ill-formed call like f(a, b, +) to be parsed as f(a + b)).

Array subscripts using square brackets can be handled in the same way as function calls.

After all this, the algorithm has grown a bit, and is a little trickier to get right in all the corner cases—but it’s still pretty simple, and certainly less work than a full-blown syntax-directed parser! In my opinion, it’s a shame that the shunting-yard algorithm isn’t more widely discussed as part of standard texts and computer-science university courses. Even the Dragon Book doesn’t so much as mention it! I never heard of this algorithm until I happened to see a forum post that referenced it, but its simplicity, elegance, and efficiency make it a superior solution for many use-cases of processing arithmetic expressions.

As you can see, even though Idyll is at a very, very early stage (it has no textures, only ambient and directional lighting), it still has a fairly complete performance measurement system. I chose to implement this early on in development because it’s my belief that although it can be too early to optimize, it’s never too early to profile.

Even at the very beginning of development, I want to know that the performance numbers I’m seeing are reasonable. I don’t need to worry about the details—I’m not going to worry that 0.47 ms is too long to spend drawing a 2700-triangle, untextured city—but I do want to know the numbers are at about the right order of magnitude. To put it another way, if I were spending 4.7 ms drawing a 2700-triangle, untextured city, then I’d be wondering what was going on! More than likely, it would be because I was doing something dumb that was forcing the driver or GPU into a slow mode of execution. This kind of bug can be hard to spot because the rendered frame is still visually correct, so you can’t see it (unless it causes you to drop from 60 to 30 Hz or something like that). But if you’re measuring your performance and you at least sanity-check your numbers from time to time, you’ll notice something’s wrong.

The other reason to implement GPU profiling early is that when it comes time to do your performance optimizations for real, you’ll have more trust in your profiling system. You’ll have ironed out most of the bugs, and you’ll have a feel for how much noise there is in the data and what sorts of artifacts there may be.

So now that you know why you should do GPU profiling ;), how do you actually do it? In this article, I’ll show you how to set it up in Direct3D 11. Similar commands should be available in any modern graphics API, although the details may vary.

First, let’s talk about how not to do GPU profiling. Do not measure performance in terms of framerate! This is something many beginners do, but it’s misleading at best, since framerate isn’t a linear measure of performance. People will make statements like “turning on XYZ dropped my framerate by 10 fps”, but this is meaningless, since a drop from 100 to 90 fps is a very different thing than a drop from 30 to 20 fps. You could clarify by reporting the fps you started from, but why bother? Just report performance results using units of real time. Milliseconds are fine, although you can go one step further and express everything in fractions of your frame budget. For instance, I’m currently targeting 60 Hz, so I have 16.67 ms to render a frame. Instead of saying that my objects rendered in 0.47 ms, I could make Idyll report that they rendered in 2.8% of a frame.

Another caveat that should be obvious: you can’t measure GPU performance by timing on the CPU. For instance, calling QueryPerformanceCounter before and after a draw call won’t measure how long the GPU took to draw your objects, only how long the graphics driver took to queue up that call in the various data structures it has under the hood. That might be useful information to have in general, but it’s not GPU profiling.

Placing Queries

The tools we’ll use to get profiling data out of the GPU are ID3D11Query objects. As described in the docs, these can be created with ID3D11Device::CreateQuery, and come in various flavors. There are two we’ll want: D3D11_QUERY_TIMESTAMP and D3D11_QUERY_TIMESTAMP_DISJOINT. You’ll need one timestamp query for each stage of your renderer that you want to profile separately (in my case, one each for shadow clear, shadow objects, main clear, and main objects), plus two more to timestamp the beginning and ending of the whole frame. You’ll also need one “timestamp disjoint” query; I’ll return to that one later.

Executing a timestamp query causes the GPU to read an internal clock and store the current value of that clock somewhere for later retrieval. This is done by calling ID3D11DeviceContext::End and passing the query object you created. (There’s no call to Begin for timestamp queries, because they don’t operate like a stopwatch—they don’t measure the elapsed time between two points; they just take a clock reading at a single point.) You’ll want to place one of these calls at the beginning of the frame, and one after each chunk of rendering work you want to profile. You’ll also need one at the very end, after your call to IDXGISwapChain::Present. Each of these is a separate ID3D11Query object, because all of these timestamps need to be collected and stored separately as the GPU chews its way through the frame.

The “timestamp disjoint” query should enclose the whole frame. This one does require paired calls to ID3D11DeviceContext::Begin (just before your first timestamp) and ID3D11DeviceContext::End (just after your last one). What does the disjoint query do? Its most important role for our purposes is to tell us the clock frequency. All the timestamps we’re going to measure are expressed as counts of ticks of some internal GPU clock…but it’s the disjoint query that tells how to convert those ticks into real time. It also has a secondary purpose: if the something went wrong with the clock, such as the counter overflowing and wrapping back to zero or the clock frequency changing during the middle of the frame (as can happen on laptops due to CPU throttling), it’ll also tell us that. However, in those cases we just have to throw our data out, as we don’t have enough information to recover accurate timings. In practice, this rarely happens, and even when it does happen it only leads to one frame of missed data, so it’s not a big deal.

To recap, here’s what a render routine might look like after all these queries have been added:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

void Render (ID3D11DeviceContext * pContext)
{
    // Begin disjoint query, and timestamp the beginning of the frame
    pContext->Begin(pQueryDisjoint);
    pContext->End(pQueryBeginFrame);
 
    // Draw shadow map
    ClearShadowMap();
    pContext->End(pQueryShadowClear);
    DrawStuffInShadowMap();
    pContext->End(pQueryShadowObjects);
 
    // Draw main view
    ClearMainView();
    pContext->End(pQueryMainClear);
    DrawStuffInMainView();
    pContext->End(pQueryMainObjects);
 
    // Display frame on-screen and finish up queries
    pSwapChain->Present(1, 0);
    pContext->End(pQueryEndFrame);
    pContext->End(pQueryDisjoint);
}

Retrieving The Data

So far we’ve set up a number of query objects that tell the GPU when and how to collect the timing data we want. Once the GPU actually renders the frame and collects the data, we need to retrieve it so we can analyze and display it.

The idea is simple enough. We just use the ID3D11DeviceContext::GetData method to see if a particular query has finished yet, and return its result if it has. As the docs say, GetData returns S_FALSE if the query is still in flight on the GPU, and S_OK when it’s done. So, we just need to write a loop that waits (an idle wait, please, not a busy wait!) until GetData returns something other than S_FALSE. Then our query data will be waiting for us!

The timestamp queries all send back a UINT64 containing—you guessed it—the timestamp value, measured in some sort of clock tick. The disjoint query returns a D3D11_QUERY_DATA_TIMESTAMP_DISJOINT struct, which contains the clock frequency and, as mentioned before, the rarely-encountered “disjoint” flag that means you have to throw out the last frame’s data because something weird happened with the clock.

So all we have to do now is subtract adjacent timestamp values to get the deltas, convert those into milliseconds and off we go! The code for this might look like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

void CollectTimestamps (ID3D11DeviceContext * pContext)
{
    // Wait for data to be available
    while (pContext->GetData(pQueryDisjoint, NULL, 0, 0) == S_FALSE)
    {
        Sleep(1);       // Wait a bit, but give other threads a chance to run
    }
 
    // Check whether timestamps were disjoint during the last frame
    D3D10_QUERY_DATA_TIMESTAMP_DISJOINT tsDisjoint;
    pContext->GetData(pQueryDisjoint, &tsDisjoint, sizeof(tsDisjoint), 0);
    if (tsDisjoint.Disjoint)
    {
        return;
    }
 
    // Get all the timestamps
    UINT64 tsBeginFrame, tsShadowClear, // ... etc.
    pContext->GetData(pQueryBeginFrame, &tsBeginFrame, sizeof(UINT64), 0);
    pContext->GetData(pQueryShadowClear, &tsShadowClear, sizeof(UINT64), 0);
    // ... etc.
 
    // Convert to real time
    float msShadowClear = float(tsShadowClear - tsBeginFrame) /
                            float(tsDisjoint.Frequency) * 1000.0f;
    // ... etc.
}

Now you have the timings in milliseconds and can do with them as you please! Display them on screen, average them over time, log them to a file, or whatever else you can think of.

Double-Buffered Queries

Back toward the beginning of this article, I said that you would need one timestamp query for each rendering stage and one disjoint query. Actually, I lied. I wanted to keep things simple while explaining the basics, but the truth is that to do a proper GPU profiler, you’ll need to double-buffer your queries, meaning that you need two copies of everything.

This is because the CPU and GPU run simultaneously, not sequentially—or at least they should, for efficiency! While the GPU is rendering frame n, the CPU is forging ahead and working on frame n + 1. When the GPU finishes frame n, it puts it up on screen and starts on frame n + 1; then the CPU gets the go-ahead to start on frame n + 2. But this means that if we fire off a bunch of timestamp queries in frame n on the CPU, then wait for them to complete before we start the next frame, we’ll be serializing the system. The CPU will have to wait for the GPU to finish the frame it just dispatched, and then the GPU will idle waiting for the CPU to finish the next frame. This is a Bad Thing.

Double-buffering allows us to get around this by keeping separate queries for alternate frames. On frame n we’ll fire off one set of queries, then we’ll wait for the other set of queries to complete—these were the queries for frame n – 1. Then we’ll re-use the queries we just collected and fire off a new set for frame n + 1, then finally wait for the queries from frame n. This way we always give the GPU a whole frame to finish its work, and avoid dropping into the slow, serialized mode of execution.

(By the way, this is a good example of what I mentioned at the top of this article about sanity-checking your performance numbers! You might have noticed that I’m measuring the total frame time on the CPU as well as on the GPU—on the CPU, it’s just QueryPerformanceCounter as normal. When the CPU and GPU are running simultaneously, as they should be, the CPU and GPU frame time will be equal (plus or minus some measurement noise). But if you didn’t double-buffer your queries, or did something else to make them drop into the slow, serialized execution mode, you’d see the CPU frame time shoot up to substantially larger than the GPU frame time, since it’s doing its own work and then waiting for the GPU to finish all its work.)

At this point, you should have all the information you’ll need to go and write a GPU profiler of your own! But in case some of the details aren’t clear, I’ve posted my GPU profiling code from Idyll, which you’re free to include in your own project; it’s under the BSD license. In that code, I’ve defined an enum called GTS (GPU Time-Stamp) to label all the timestamped points in the frame; you should modify it to contain the set of blocks you care about. It handles all the double-buffering for you, and will also average all the deltas over a half-second, for smoother display. Now, go forth and find out how long the GPU is taking to render your scenes!

My primary goals with this project are:

1. To teach myself the Direct3D 11 API. Since my background is in OpenGL 3.0 and PS3 (which has an API similar to D3D9), I want to update my knowledge. I actually started this project with D3D 10, but I intend to convert it to D3D 11 soon.
2. To have a place to experiment with new graphics techniques a little more freely than I might in the context of my day job. The workplace is great for some sorts of things, but playing around with ideas just to see where they go is not one of them. :)

That’s the general idea for where I’m going with this. Just to give you something concrete, here are a couple of screenshots of Idyll in its current state. I wasn’t kidding when I said it’s just in its infancy! It doesn’t have textures or any materials other than Lambert, and the scene is just something I threw together in an afternoon.

I apologize for the shadow quality; it’s just a single 1024² shadow map stretched across the whole scene. The lighting is a directional light, plus an ambient with a vertical ramp from blue to black. It is lit in linear color space, though!