Thanks for Benjamin Larsson for advising me on how to make a proper graph using Graphviz (specify a graph of type ‘prof’ vs. ‘unix’). Thus, here are some nice (and big) graphs based on the call/ret experiment at this point:

• wmv2init() callgraph
• wmv2packet() frame #0 decode callgraph
• wmv2packet() frame #18 decode callgraph

See this post (“Refining The call/ret Monitor”) for details on how the graph data is generated.

And as a bonus, I put the toolset to work analyzing Linspire’s WMV3/WMV9 decoding module. Here are some call graphs (I used the teaser trailer from Halo 2, hence the “halo2” in the filenames):

• wmv3init() callgraph; I like the way the graph expresses countless functions invoking wmvMalloc() which simply calls malloc()
• wmv3packet() frame #0 decode callgraph; note that this is just a plain, black I-frame and does not exercise a lot of the logic in the module; let’s start small

Reverse engineering is, of course, a tedious, time-consuming, and error-prone task. It requires a lot of concentration that I either do not have or do not care to invest in the RE task. That is why so many of my RE experiments are geared toward automating the task as much as possible. To that end, I am optimistic about this call/ret monitoring experiment since it yields such a good high-level overview of an algorithm contained in a binary (with debug symbols). But it can use some improvement(s):

Continue reading →