Breaking Eggs And Making Omelettes

Topics On Multimedia Technology and Reverse Engineering


Archives:

Using lcov With FFmpeg/Libav

November 20th, 2011 by Multimedia Mike

Last year, I delved into code coverage tools and their usage with FFmpeg. I learned about using GNU gcov, which is powerful but pretty raw about the details it provides to you. I wrote a script to help interpret its output and later found another script called gcovr to do the same, only much better.

I later found another tool called lcov which is absolutely amazing for understanding code coverage of your software. I’ve been meaning to use it to further FATE test coverage for the multimedia projects.



Click for larger image

Basic Instructions
Install the lcov tool, of course. In Ubuntu, 'apt-get install lcov' will do the trick.

Build the project with code coverage support, i.e.,

./configure --enable-gpl --samples=/path/to/fate/samples \
 --extra-cflags="-fprofile-arcs -ftest-coverage" \
 --extra-ldflags="-fprofile-arcs -ftest-coverage"
make

Clear the coverage data:

lcov --directory . --zerocounters

Run the software (in this case, the FATE test suite):

make fate

Let lcov work its magic:

lcov --directory . --capture --output-file coverage.info
mkdir html-output
genhtml -o html-output coverage.info

At this point, you can aim your web browser at html-output/index.html to learn everything you could possibly want to know about code coverage of the test suite. You can sort various columns in order to see which modules have the least code coverage. You can drill into individual source files and see highlighted markup demonstrating which lines have been executed.

As you can see from the screenshot above, FFmpeg / Libav are not anywhere close to full coverage. But lcov provides an exquisite roadmap.

Posted in Programming | 8 Comments »

What’s So Hard About Building?

September 9th, 2011 by Multimedia Mike

I finally had a revelation as to why so building software can be so difficult– because build systems are typically built on programming languages that you don’t normally use in your day to day programming activities. If the project is simple enough, the build system usually takes care of the complexities. If there are subtle complexities — and there always are — then you have to figure out how to customize the build system to meet your needs.

First, there’s the Makefile. It’s easy to forget that the syntax which comprises a Makefile pretty well qualifies as a programming language. I wonder if it’s Turing-complete? But writing and maintaining Makefiles manually is arduous and many systems have been created to generate Makefiles for you. At the end of the day, running ‘make’ still requires the presence of a Makefile and in the worst case scenario, you’re going to have to inspect and debug what was automatically generated for that Makefile.

So there is the widespread GNU build system, a.k.a., “the autotools”, named due to its principle components such as autoconf and automake. In this situation, you have no fewer than 3 distinct languages at work. You write your general build instructions using a set of m4 macros (language #1). These get processed by the autotools in order to generate a shell script (language #2) called configure. When this is executed by the user, it eventually generates a Makefile (language #3).

Over the years, a few challengers have attempted to dethrone autotools. One is CMake which configures a project using its own custom programming language that you will need to learn. Configuration generates a standard Makefile. So there are 2 languages involved in this approach.

Another option is SCons, which is Python-based, top to bottom. Only one programming language is involved in the build system; there’s no Makefile generated and run. Until I started writing this, I was guessing that the Python component generated a Makefile, but no.

That actually makes SCons look fairly desirable, at least if your only metric when choosing a build system is to minimize friction against rarely-used programming languages.

I should also make mention of a few others: Apache Ant is a build system in which the build process is described by an XML file. XML doesn’t qualify as a programming language (though that apparently doesn’t stop some people from using it as such). I see there’s also qmake, related to the Qt system. This system uses its own custom syntax.

Posted in Programming | 15 Comments »

Started Programming Young

September 5th, 2011 by Multimedia Mike

I have some of the strangest memories of my struggles to jump into computer programming.

Back To BASIC
I remember doing some Logo programming on Apple II computers at school in 5th grade (1987 timeframe). But that was mostly driving turtle graphics. Then I remember doing some TRS-80 BASIC in 7th grade, circa 1989. Emboldened by what very little I had learned in perhaps the week or 2 we took in a science class to do this, I tried a little GW-BASIC on my family’s “IBM-PC compatible” computer (they were still called that back then). I still remember what my first program consisted of. Even back then I was interested in manipulating graphics and color on a computer screen. Thus:

10 color 1
20 print "This is color 1"
30 color 2
40 print "This is color 2"
...

And so on through 15 colors. Hey, it did the job– it demonstrated the 15 different colors you could set in text mode.

What’s FOR For?
That 7th grade computer unit in science class wasn’t very thick on computer science details. I recall working with a lab partner to transcribe code listings into a computer (and also saving my work to a storage cassette). We also developed form processing programs that would print instructions to input text followed by an “INPUT I$” statement to obtain the user’s output.

I remember there was some situation where we needed a brief delay between input and printing. The teacher told us to use a construct of the form:

10 FOR I = 1 TO 20000
20 NEXT I

We had to calibrate the number based on our empirical assessment of how long it lasted but I recall that the number couldn’t be much higher than about 32000, for reasons that would become clearer much later.

Imagine my confusion when I would read and try to comprehend BASIC program code I would find in magazines. I would of course see that FOR..NEXT construct all over the place but obviously not in the context of introducing deliberate execution delays. Indeed, my understanding of one of the fundamental building blocks of computer programming — iteration — was completely skewed because of this early lesson.

Refactoring
Somewhere along the line, I figured out that the FOR..NEXT could be used to do the same thing a bunch of times, possibly with different values. A few years after I had written that color program, I found it again and realized that I could write it as:

10 for I = 1 to 15
20 color I
30 print I
40 next I

It still took me a few more years to sort out the meaning of WHILE..WEND, though.

Posted in Programming | 3 Comments »

The Fastest Way To Learn Assembly Language

September 3rd, 2011 by Multimedia Mike

I saw an old StackOverflow thread linked from Hacker News asking how to whether it’s worthwhile to learn assembly language and how to go about doing so. I’d like to take a stab at the last question.

The fastest way to learn an assembly language is to reverse engineer something. Seriously, start with something that you know (like a C program that you wrote yourself) and take it apart. The good news is that assembly language is very simple and you will get a lot of practice in a short amount of time with RE.

So here’s how you do it:

  • Take a simple program in C and build it with your tool chain, whether GNU gcc on Linux, Xcode on Mac, or MSVC on Windows. Also, make sure to turn on debugging symbols during compilation (this will help annotate the disassembly).
  • On Linux, use objdump: objdump -d program_binary
  • On Mac, use otool: otool -tV program_binary
  • On Windows: I admit, I’m a bit fuzzy on this one– I’m quite certain there’s a standard MSVC tool that prints the assembly listing.

Anyway, look at the disassembled code and find the main() function. Work from there. Whatever the first instruction is, look it up on Google. You’ll likely find various CPU manuals that will explain the simple operation of the instruction. Look up the next unfamiliar instruction, then the next. Trust me, you’ll become an ASM expert in no time.

Good luck!

Posted in Programming | 6 Comments »

« Previous Entries Next Entries »