Category Archives: Reverse Engineering

Brainstorming and case studies relating to craft of software reverse engineering.

The Almighty EULA

One of my favorite blogs, Coding Horror, recently had an intelligent discussion of end-user license agreements (EULAs). When I say “intelligent”, I of course mean that it doesn’t credit them as being the ultimate evil in the world that will enslave us all.


EULA Hotel
It’s a real hotel in San Francisco! Thanks to King Molan for the picture.

Seriously, I think EULAs are very important. Many EULAs contain clauses that forbid binary reverse engineering. Whenever I install a piece of closed, proprietary software, I skim the EULA specifically to locate the section that discusses RE and the forbidding thereof. My reasoning is that if the clause is missing, then the software’s creators may just have the source available for download somewhere which would make any RE task superfluous.

Remember, don’t RE if you don’t have to.

call/ret, Without The ret

I resurrected my old call/ret experiments recently. I wish I had published more about how I did what I did 2 years ago — that’s why I started this blog, as a research journal, for my own benefit, for when I eventually return to a project after a lengthy hiatus. Fortunately, I emailed pretty thorough details to a fellow reverse engineer during the original project and still have the old email in the archives.

To refresh, the call/ret monitor watches a piece of code run and logs when function call or return instructions are executed, and where. After the fact, and combined with executable code that comes with meaningful symbol names, it generates a call graph to demonstrate where code really flows and gets around the problem of resolving indirect jumps in static disassembly listings. The original experiment assumes that for every call, there is an equal and opposite return (formally stated: num_calls == num_rets). The target I’m after right now behaves… differently. I am seeing way more calls than returns.

Obviously, there is a lot about compilers that I don’t understand. I think what I’m looking at in this situation is an artifact of C++ or perhaps its linkage from C. When I examine functions in static disassembly, there will be 7 NOP instructions after the stack frame initialization. Seems pretty innocuous. However, when I analyze the code more carefully while it’s running, I see that something — I suspect the dynamic loader — is using those 7 NOP bytes for something else. I imagine this is some common convention, and I would love to know more about it someday. But for the time being, I just want to get past this problem. In this particular situation, all that the patched code does is perform a call 5 bytes farther in the address space where a pop instruction is waiting to clear the return address that just got pushed on the stack by the call. So that’s where the massive call surplus comes from. Why the unknown entity couldn’t patch in an absolute jump is unclear.

Eventually, it dawned on me that the /ret half of the call/ret experiment was entirely unnecessary given the other data I already have. I have retooled the experiment with that in mind. It’s annoying to try to remember enough Perl to make this fly but it’s worth it for the resulting data visualization.

ATRAC3 Decoder

Ever so quietly, a new open source ATRAC3 decoder implementation has been slipped into FFmpeg. This decoder handles atrc data inside of RealMedia files or in WAV files.

Thanks to Benjamin Larsson and Maxim Poliakovski for their diligent work on this, as well as the guru for his tireless reviewing efforts and uncompromising code quality standards.

RealAudio samples here and WAV samples here.

Classical Audio

Trixter, digital P.I. and archaeologist, has been tracking down authors who long ago reverse engineered custom audio formats. He stalked down one Adrienne Cousins, the author of a program called Sputter, a multi-format audio file encoder/decoder… all in 16-bit x86 ASM! The source includes codecs for the Covox ADPCM formats mentioned recently. Thanks to Kostya, though, for independently reverse engineering and documenting the format.

Still, there might be some other gems in the Sputter source code. The author has granted permission to redistribute the source, and it is now in this directory: http://samples.mplayerhq.hu/A-codecs/CreativeADPCM8bit/. Also in that directory is a package called VEDIT.rar which contains an old EGA-based program that is able to encode VOC files to that arcane 2.6-bit ADPCM format that the Sound Blaster could encode and decode in hardware. If reversed engineered, it might describe how the 2.6-bit algorithm works.