Nathan Reed

Blog Stuff I’ve Made Talks About Me
Quick And Easy GPU Random Numbers In D3D11Greg Egan’s Orthogonal: Alternate Physics, Familiar Politics

Visual Studio Debug Visualizer For Half Floats

February 6, 2013 · Coding · Comments

Here’s something handy I managed to put together the other night: a Visual Studio autoexp.dat debugger visualizer for half-precision floats. It’s written against the half type from the OpenEXR library, but if you have a custom half type it should be easy to adapt to that. It handles infinities, NaNs, and denormals.

half{
    preview(
        #if (($e._h & 0x7c00) == 0x7c00) (
            #if (($e._h & 0x03ff) == 0) (
                #if ($e._h & 0x8000) ("-inf") #else ("+inf")
            ) #else (
                #if ($e._h & 0x0200) ("qNaN") #else ("sNaN")
            )
        ) #else (#if (($e._h & 0x7c00) == 0) (
            ; Denormal or zero
            [($e._h & 0x03ff) / 16777216.0 * (($e._h >> 0xf) * -2.0 + 1.0), g]
        ) #else (
            ; Normal float (condensed to avoid length problem)
            [(1<<(($e._h&0x7c00)>>0xa))/32768.0*(1.0+($e._h&0x03ff)/1024.0)*(($e._h>>0xf)*-2.0+1.0),g]
        ))
    )
}

Paste this into the [Visualizer] section of your autoexp.dat. I’ve tested it in Visual Studio 2008 and 2010.

Unfortunately, in both, there seems to be a limit of around 100 characters (I’m not sure of the exact value) on the length of the contents of the square-bracket formatting operator; if you go past the limit it will error out and simply ignore your visualizer. This only affects the Local variables window, but not the Watch window (or perhaps the limit is longer in the Watch window). I had to remove all the whitespace from the normal-float case to make it work.

Also, as the comments to autoexp.dat warn, if hex mode is turned on in the debugger then any int literals in the visualizer code will be interpreted as hex, if they aren’t already. Hence all the ints are written in hex to avoid this problem.

Here it is in action:

Half-float visualizer in action

Tweet
Quick And Easy GPU Random Numbers In D3D11Greg Egan’s Orthogonal: Alternate Physics, Familiar Politics

Comments on “Visual Studio Debug Visualizer For Half Floats”

Subscribe

  • Follow in Feedly Feedly
  • RSS RSS

Recent Posts

  • Reading Veach’s Thesis, Part 2
  • Reading Veach’s Thesis
  • Texture Gathers and Coordinate Precision
  • git-partial-submodule
  • Slope Space in BRDF Theory
  • Hash Functions for GPU Rendering
  • All Posts

Categories

  • Graphics(32)
  • Coding(23)
  • Math(21)
  • GPU(15)
  • Physics(6)
  • Eye Candy(4)
© 2007–2023 by Nathan Reed. Licensed CC-BY-4.0.