Breaking Eggs And Making Omelettes

Topics On Multimedia Technology and Reverse Engineering


Archives:

Installing CrystalHD Drivers In Linux

January 5th, 2010 by Multimedia Mike

Executive Summary: I tried to get a Broadcom CrystalHD chip to work in Linux. I came close to being successful. The chip, kernel driver, and userspace library all work. The example app that would have been the payoff… not so much. I document my process in this post in case others need assistance, or can lend assistance in the final step.

There was some news recently about Broadcom open sourcing code related to a video decoding chip. The brand name here is apparently “CrystalHD” and the chip in question is the BCM-70012. I came into possession of one of these and endeavored to make the open source software surrounding it work in Linux.

The first issue is installing the hardware. It’s a PCI Express mini card which has the same form factor as a PCIe mini wireless networking card. So if your computer can host such a wireless card, it can also hold this thing. Allegedly. First, I tried to place it in the empty PCIe-mini slot in my MSI Wind Nettop. No go. The machine refused to boot up (it would power up but never beep to indicate that it’s really ready to run). Removing the card made the problem go away.

So determined was I to make this chip work that I actually took apart my dear Eee PC 701, ripped out the wireless card and replaced it with the Broadcom card. Deciding it would be too much trouble to attempt to re-attach the keyboard and touchpad ribbons, I realized I could just use USB peripherals.

Resigned to the notion that I just foolishly destroyed my 2 year old Eee PC, I threw the switch anyway and was quite surprised to see it boot up normally. An ‘lspci’ command indicates a new Broadcom multimedia controller hanging off the PCI bus. It’s not pretty but it’s breathing:


Eee PC 701 disassembled with Broadcom CrystalHD decoder installed

So let’s talk software. Broadcom released the driver as open source. To many in the open source community, this is tantamount to, “Okay, done deal! What else needs to be open sourced?” Not so fast. There’s no documentation in the whole package (user-wise, anyway; the libcrystalhd API is thoroughly documented in header comments). So I will describe my experiences with the software.

Download the Linux driver package and unpack it.

Have some kernel source to build against. You don’t actually need a full source tree installed; just the headers will do. To install the appropriate header package on an Ubuntu system, execute:

  apt-get install linux-headers-`uname -r`

When installing a linux-headers package, it’s necessary to specify a kernel version. `uname -r` will determine your Linux kernel version and substitute it into the command line.

If you’re going the route of installing only kernel headers like I did, you will also need to create a symlink from /lib/modules/<version>/build to the new headers tree. This should do the trick:

  ln -s /usr/src/linux-headers-`uname -r` /lib/modules/`uname -r`/build

With that infrastructure in place, change directory into crystalhd/driver/linux. Important step: Run ‘dos2unix *’ in this directory. There are DOS-style line endings running around in here that will foul up the build process. Run ‘autoconf’. Then run ‘./configure’. The whole process looks sort of like this:

$ dos2unix *
$ autoconf 
$ ./configure 
checking for ld... ld
configure: creating ./config.status
config.status: creating ./Makefile

Time to build: ‘make’. This should look something like:

$ make
make -C /lib/modules/2.6.30.5-ep0/build SUBDIRS=/home/melanson/crystalhd/driver/linux modules
make[1]: Entering directory `/usr/src/linux-headers-2.6.30.5-ep0'
  CC [M]  /home/melanson/crystalhd/driver/linux/crystalhd_lnx.o
  CC [M]  /home/melanson/crystalhd/driver/linux/crystalhd_misc.o
  CC [M]  /home/melanson/crystalhd/driver/linux/crystalhd_cmds.o
  CC [M]  /home/melanson/crystalhd/driver/linux/crystalhd_hw.o
  LD [M]  /home/melanson/crystalhd/driver/linux/crystalhd.o
  Building modules, stage 2.
  MODPOST 1 modules
  CC      /home/melanson/crystalhd/driver/linux/crystalhd.mod.o
  LD [M]  /home/melanson/crystalhd/driver/linux/crystalhd.ko
make[1]: Leaving directory `/usr/src/linux-headers-2.6.30.5-ep0'

‘make install’ (as root, of course) to get the module where it needs to go and then ‘modprobe crystalhd’ to activate the driver. After the modprobe command, the last line of my ‘dmesg’ log looks like:

  Broadcom 70012 Decoder 0000:01:00.0: setting latency timer to 64

We’re not done yet, though. There are 2 provided scripts (bcm_70012_???.sh) which are supposed to load the module and create a device node. The scripts failed to run. Instead, I sorted out the right command for my machine:

mknod -m 666 /dev/crystalhd c 251 0

The major number (251) comes from ‘grep crystalhd /proc/devices’.

Then we move over to the user-land system library (crystalhd/linux_lib/libcrystalhd). A simple ‘make && sudo make install’ should do the trick here.

Then there’s the matter of the example code (crystalhd/examples). Whoever wrote these samples was not in close communication with the people who wrote the rest of the library. It’s reasonable that the code did work at some point. But whoever made this final package reorganized the directory structure and didn’t test this code since there are all kinds of problems when running the simple build script included.

Things went off the rails at this point.

The build script doesn’t build as-is due to incorrect paths. I changed the build script to look like this:

g++ -I ../include/ -I ../linux_lib/libcrystalhd/ -I/usr/include -D__LINUX_USER__ -lcrystalhd -lpthread hellobcm.cpp -o hellobcm

This is to be run from the examples directory. Doing so, however, indicates that hellobcm.cpp has several problems. Upshot: I altered the start of the file to look like this (added 2 headers, subtracted one, added U8 and U32 typedefs which might only work on x86_32):

#include <stdlib .h>
#include <stdio .h>
#include <stdint .h>
#include <string .h>
#include <semaphore .h>
#include "bc_dts_types.h"
#include "bc_dts_defs.h"
#include "libcrystalhd_if.h"
//#include "bc_ldil_if.h"
#include <iostream>
#include <fstream>
#include <sys /shm.h>

typedef unsigned char U8;
typedef unsigned int U32;

#define TRY_CALL_1(func, p1, errmsg) \
[...]

That gets the test program to compile. Running it results in a complaint that firmware can’t be located in a preordained location:

[...]
DtsGetFirmwareFiles:Ctx->FwBinFile is /lib/firmware/bcmFilePlayFw.bin
Failed to Open FW file
[...]

All right, so it wants the firmware files in very specific places (this is actually due to that libcrystalhd user-land library). These firmware files live in crystalhd/firmware/fwbin/70012. Moving the firmware files to /lib/firmware (hey! there are a bunch of firmware files there; I never knew that), makes hellobcm get farther before tossing out a cryptic error. A little investigation reveals that the source is looking for a hardcoded .264 video file (home directory paths implicate a ‘jarod’ and a ‘davilla’, perhaps the same davilla who authored the XBMC blog post).

So I need to feed it a raw .264 video file. All right, Broadcom, let’s see what you’ve got. Chew on the MV1_BRCM_1 H.264 sample (src19td.IBP.264 from here), one of the meanest conformance samples in the ITU suite. Incidentally, it’s from Broadcom (hence, “BRCM”).

Things go… well, confusingly. The program emits a lot of debugging info which largely emanates from the libcrystalhd.so library (that’s also the code which downloads the firmware at initialization). I’m still massaging the code (various other parameters seem to be hardcoded based on the sample) but when it does run successfully, it claims to decode 136/257 frames from the sample. And it does so while occupying the CPU for about a quarter of a second.

For comparison, it takes FFmpeg 16.6 seconds of CPU time to decode this sample using the following command line:

  ffmpeg -benchmark -i src19td.IBP.264 -f yuv4mpegpipe -y /dev/null

It is, of course, inappropriate to make comparisons before I can validate that this example is decoding anything correctly (and it’s a specious comparison anyway). It should be noted that 16.6s amounts to “less than realtime” for a sample that’s 10 seconds long. So that’s the time to beat on my Eee PC 701 equipped with an Intel Celeron M CPU running at 620 MHz… wait, no, ACK! /proc/cpuinfo reports the full 900 MHz! I accidentally overclocked the thing. Or rather, de-underclocked it. Oops. Per my understanding, the only negative implication of that is less battery time.

This same CrystalHD code seems to have been copy and pasted for crystalhd-for-osx. The same example programs live in that tree. I doubt they compile.

Would CrystalHD driver support be useful for FFmpeg? It doesn’t really seem like a good fit. But then, FFmpeg already supports other hardware decoding features such as XvMC and VDPAU.

Posted in Open Source Multimedia | 16 Comments »

16 Responses

  1. Diego E. "Flameeyes" Pettenò Says:

    Interesting; I’d be curious of playing with this myself, but I really don’t have the time.

    I’m not sure though, which license is the whole shebang released under? Because I’m pretty sure the Linux developers won’t like both the external modules and the firmware being loaded from userspace rather than the kernel itself (which is where most of the firmware gets loaded nowadays), and the lack of auto-generation of device nodes through udev.

    But at least it’s a start I guess. Do you know if there is any way at all to have those things working on a standard desktop system?

  2. Multimedia Mike Says:

    Allegedly, there exist PCIe adaptors for PCIe mini cards which allow their use in larger systems.

    As for licenses, all of the kernel driver source files have a GPL 2 license header. The libcrystalhd stuff is all LGPL 2.1. The example source is ambiguous (and not working anyway).

  3. Multimedia Mike Says:

    Also, there is apparently movement on getting the Broadcom driver into the official kernel tree: http://www.gossamer-threads.com/lists/linux/kernel/1173758

  4. scott davilla Says:

    Hi Mike,

    see http://git.wilsonet.com/crystalhd.git/ , this is public upstream. There were some opps in pushing this code public. Yes we are working directly with Broadcom MediaPC group.

    crystalhd/examples was a quick and dirty so that users can see how to use the lib API. The process of pushing back to Broadcom mangled it a bit.
    H.264 needs to be in annex B format (byte stream instead of bit-stream), there and ffmpeg demuxer filter that will do this. m2ts and mpeg2 demuxer packets from ffmpeg will pass fine.

    Feel free to email and ask questions. davilla [at] xbmc [dot] org

  5. scott davilla Says:

    @Diego

    driver is gplv2, library is lgplv2 as is gstreamer code. firmware is closed source. You are correct, library loads the firmware and it’s actually the best way to handle closed source firmware. There’s no reason for the kernel driver to “own” the firmware. This is a userland video decoder and there’s no reason to stuff all this into a kernel driver, it’s a waste of kernel mem.

    See the git public upstream tree. There are many additions added to the original release code including a udev rule.

  6. Multimedia Mike Says:

    Thanks for your attention, davilla. I checked out the current tree. There isn’t much different vs. the Broadcom release from 2009-12-29. One key difference is the firmware file that’s being download. And everything has proper line endings now. Also, the hellobcm.cpp program compiles without modification now. It still doesn’t work for me, however.

    The sample file that you and Jarod are using — test_video.264 — is that a JVT NAL sequence as reported by ‘file’?

  7. scott davilla Says:

    yes, test_video.264 is a JVT NAL sequence

  8. Jarod's Junk Collection » The Broadcom Crystal HD and Linux story thus far… Says:

    […] want to poke at it and need help making it behave, but most of the gory details can be seen in the post and comments over here in the Breaking Eggs And Making Omelettes blog, which if I understand correctly, belongs to one of the ffmpeg […]

  9. Dan Says:

    Hello,

    I just went through this exercise with the Broadcom PCI mini card. The driver builds and loads. The hellobcm program runs, although it continuously fails. If you kill the process midstream, it locks up the whole computer. Libva doesn’t seem to recognize this card either, but mostly it complains about not being able to open DRI2 (isDRI2Connected => fail), and mplayer-accelerated with -vo vaapi failes when it tries to open DRI2. So not much luck here getting this thing working:

    ./hellobcm
    starting up
    Running DIL (0.9.25) Version
    Mdata Pool Created…
    DtsSetupHardware: DtsPushAuthFwToLink
    DtsGetFirmwareFiles:Ctx->FwBinFile is /lib/firmware/bcmFilePlayFw.bin
    DtsSetupHardware: Success
    ADD buffs
    …..
    ADD buffs
    DbgOptions=b0000025
    try calls done
    file opened successfully
    Input Buffer: 0x812db20
    Aligned Input Buffer: 0x812db20, Offset = 0
    Y Buffer: 0xb53b3008
    DtsFetchOutInterruptible: Failed:a
    Timeout in DtsProcOutput. Accum Bytes: 65536
    ..etc.

  10. Gavin Stark Says:

    I get the exact same results as Dan above. This is with the code crystalhd_linux_20091229.zip

  11. Gavin Stark Says:

    Update, when I downloaded a sample file from here: http://code.google.com/p/crystalhd-for-osx/downloads/detail?name=test_video.264 the hellobcm file could process it. (at least it stopped generating errors)

  12. Multimedia Mike Says:

    @Gavin: I had a similar experience with that updated code and the specific test sample. Regrettably, the code doesn’t allow easy validation of the results.

  13. Gavin Stark Says:

    OK, so now that I could confirm that the “hellobcm” program would process that very specific test file (hopefully confirming all the drivers/libraries are working) I thought I’d try the gstreamer plugin. Even against that sample file it fails to decode it. gstreamer WILL play the file without the broadcom plugin available. Anyone have success getting the gstreamer plugin working?

  14. julio De Gregorio Says:

    Hi, I wonder if you could get the failure to boot fixed. I am experiencing the same on an eee box b202.
    Although plugging the card after boot works if you reboot. Problem is only showing up when COLD booting.
    The card works fine with XBMC
    My problem is I have to remove and replace the card every power off or power failure =(
    Any help is welcome.

  15. mark Says:

    I have A crystal hd 7925 not working in ubuntu 10.10. I ound his pae but for rge number in

    mknod -m 666 /dev/crystalhd c 251 0

    I run
    grep crystalhd /proc/device
    ?
    this yields a blank result. No what?

    You also refer to “user land files ” can you please clarify.

  16. mark Says:

    sorry that was suppsd to have been 7015 . Small keyboard kills me