Visual Studio Debug Visualizer For Half Floats

February 6, 2013 · Practical · No comments

Here’s something handy I managed to put together the other night: a Visual Studio autoexp.dat debugger visualizer for half-precision floats. It’s written against the half type from the OpenEXR library, but if you have a custom half type it should be easy to adapt to that. It handles infinities, NaNs, and denormals.

half{
    preview(
        #if (($e._h & 0x7c00) == 0x7c00) (
            #if (($e._h & 0x03ff) == 0) (
                #if ($e._h & 0x8000) ("-inf") #else ("+inf")
            ) #else (
                #if ($e._h & 0x0200) ("qNaN") #else ("sNaN")
            )
        ) #else (#if (($e._h & 0x7c00) == 0) (
            ; Denormal or zero
            [($e._h & 0x03ff) / 16777216.0 * (($e._h >> 0xf) * -2.0 + 1.0), g]
        ) #else (
            ; Normal float (condensed to avoid length problem)
            [(1<<(($e._h&0x7c00)>>0xa))/32768.0*(1.0+($e._h&0x03ff)/1024.0)*(($e._h>>0xf)*-2.0+1.0),g]
        ))
    )
}

Paste this into the [Visualizer] section of your autoexp.dat. I’ve tested it in Visual Studio 2008 and 2010.

Unfortunately, in both, there seems to be a limit of around 100 characters (I’m not sure of the exact value) on the length of the contents of the square-bracket formatting operator; if you go past the limit it will error out and simply ignore your visualizer. This only affects the Local variables window, but not the Watch window (or perhaps the limit is longer in the Watch window). I had to remove all the whitespace from the normal-float case to make it work.

Also, as the comments to autoexp.dat warn, if hex mode is turned on in the debugger then any int literals in the visualizer code will be interpreted as hex, if they aren’t already. Hence all the ints are written in hex to avoid this problem.

Here it is in action:


Leave a Reply

Your email address will not be published. Required fields are marked *