Author Archives: Multimedia Mike

call/ret, Without The ret

I resurrected my old call/ret experiments recently. I wish I had published more about how I did what I did 2 years ago — that’s why I started this blog, as a research journal, for my own benefit, for when I eventually return to a project after a lengthy hiatus. Fortunately, I emailed pretty thorough details to a fellow reverse engineer during the original project and still have the old email in the archives.

To refresh, the call/ret monitor watches a piece of code run and logs when function call or return instructions are executed, and where. After the fact, and combined with executable code that comes with meaningful symbol names, it generates a call graph to demonstrate where code really flows and gets around the problem of resolving indirect jumps in static disassembly listings. The original experiment assumes that for every call, there is an equal and opposite return (formally stated: num_calls == num_rets). The target I’m after right now behaves… differently. I am seeing way more calls than returns.

Obviously, there is a lot about compilers that I don’t understand. I think what I’m looking at in this situation is an artifact of C++ or perhaps its linkage from C. When I examine functions in static disassembly, there will be 7 NOP instructions after the stack frame initialization. Seems pretty innocuous. However, when I analyze the code more carefully while it’s running, I see that something — I suspect the dynamic loader — is using those 7 NOP bytes for something else. I imagine this is some common convention, and I would love to know more about it someday. But for the time being, I just want to get past this problem. In this particular situation, all that the patched code does is perform a call 5 bytes farther in the address space where a pop instruction is waiting to clear the return address that just got pushed on the stack by the call. So that’s where the massive call surplus comes from. Why the unknown entity couldn’t patch in an absolute jump is unclear.

Eventually, it dawned on me that the /ret half of the call/ret experiment was entirely unnecessary given the other data I already have. I have retooled the experiment with that in mind. It’s annoying to try to remember enough Perl to make this fly but it’s worth it for the resulting data visualization.

VQ/RoQ Coding Exercise

Once upon a time, a talented developer named Eric Lasota wrote a RoQ encoder named SwitchBlade. He also made a patch to hook it up to FFmpeg. Your job, if you are searching for an entry level task for jumping into FFmpeg development, is to revise that patch for inclusion into the main FFmpeg source tree. All the code is there. I think that the main chores involved will be reformatting the code to use the same style as FFmpeg, and concatenating the source files in a sane manner. You may need to revise some of the code per the guru’s specifications. Email me (email address under links on side bar) for more advice if you would like to take on this task.

Check his projects page to find the project source.

VQ Case Study: Textures

Per my understanding, a lot of 3D hardware operates by allowing the programmer to specify a set of vertices between which the graphics chip draws lines. Then, the programmer can specify that a bitmap needs to be plotted between some of those lines. In 3D graphics parlance, those bitmaps are called textures. More textures make a game prettier, but a graphics card only has so much memory for storing these textures. In order to stretch the video RAM budget, some graphics cards allow for compressing textures using vector quantization.

A specific example of VQ in 3D graphics hardware is the Sega Dreamcast with its PowerVR2 graphics hardware. Textures can be specified in a number of pixel formats including, but not limited to, RGB555, RGB565, and VQ. In the VQ mode, a 256-entry vector codebook is initialized somewhere in video RAM. Each vector is 8 bytes large and specifies a 2×2 block of pixels in either RGB555 or RGB565 (can’t remember which, or it might be configurable). For the texture in video RAM that is specified as VQ, each byte is actually an index into the codebook. Instant 8:1 compression, notwithstanding the 2048-byte codebook overhead which can be negligible depending on how many textures leverage the codebook and how large those textures are.

VQ Case Study: RoQ

RoQ was first developed for the FMV-based adventure game The 11th hour and was later adopted by Id for the Quake III engine and derivative games.

RoQ operates in a YUV 4:2:0 space. However, it was developed for a game released in the late 1994/early 1995 timeframe. Back then, cutting edge video was 640×480 at 256 colors or maybe 64K colors. and it was not feasible to take a large video frame and convert the entire thing from YUV -> RGB 30, 24, or even 15 times per second. However, RoQ’s design solved some of these problems.

Continue reading →