Skip to content

Broadcom CrystalHD Decoder support for FFmpeg and MPlayer

At the end of last year, Broadcom released open-source drivers and a library for their CrystalHD hardware video decoder; You can read the details about that at Jarod Wilson’s blog if you’re interested.

The hardware is particularly attractive because it’s low cost and can be added to any system, regardless of the GPU it uses. It provides MPEG1/2, H.264 and VC-1 decode capabilities in all hardware versions, and the latest 70015 part also adds MPEG4 Part 2 / DivX / XviD support – and, if you care about such things, it does so in a way that means all the infamous patent issues are handled in hardware.

Once the drivers and library were released, we started to see plenty of application support emerge, with XBMC, Xine and MythTV drivers under development early on, and a gstreamer plugin provided by Broadcom with the library. In the last couple of months, a VLC patch has been proposed. But, for all this, there was no movement on FFmpeg or Mplayer; the first being the most widely used codec library around and the later, one of the most widely used media players (and obviously a consumer of FFmpeg). I bought myself a 70012 and later a 70015 with the intention of playing around with them, but when I got some free time last month, I started working on FFmpeg and Mplayer support.

After some experimentation, I did the initial implementation as a native Mplayer decoder, which helped remove FFmpeg as a variable while I tried to get things working correctly, and after some effort, I came up with something that worked pretty well for progressive content. and partially worked for some interlaced content. In my initial discussion on the mplayer mailing list, it became clear that it needed to be an FFmpeg decoder to be maintainable for the long term. So I went back and converted it, which was relatively straight-forward as the APIs are very similar. I’ve now started getting review feedback on the FFmpeg list, and I expect it will be quite a while before it gets in, assuming it ever does, but the code is definitely usable enough to publicise more widely.

What Works

  • MPlayer playback
  • 70015 Hardware: This is the newest part, with extra codec support
  • All the officially supported content types except DivX 3.11
  • Progressive Content
  • Interlaced MPEG2 and H.264 MBAFF Content

You’ll note that I said MPlayer playback works but nothing about FFmpeg even though it’s an FFmpeg decoder. The issue here is that the hardware is pipelined, so it is normal for many frames to be in flight at one time – perhaps as many as 20 under normal conditions, and over a hundred if things start going wrong… This means that the application has to have a concept of lag in the decoding process. Now, while Mplayer uses FFmpeg’s codecs, it handles frame timing and av sync in a completely different way from the actual FFmpeg transcoding application, and it happens to do it in a way that can cope with the hardware behaviour (although I had to increase the number of frames in flight that Mplayer can handle). FFmpeg on the other hand assumes that when it submits a frame for decoding, the frame it gets back is the same one. From the discussion, it sounds like I’ll need to add the same mechanism that Mplayer uses to FFmpeg. So, for now, it’s an FFmpeg decoder that doesn’t work with FFmpeg :-)

The 70012 is the previous generation of hardware and it has some significant differences from the 70015. Beyond the reduced codec support, it has much tighter requirements with respect to keeping the pipeline happily fed, and I haven’t had a chance to investigate what this really means yet. For now, it will kind-of work, but expect something to go wrong with the pipeline and either get input overflow or output underflow pretty quickly.

Interlaced content is tricky because it can appear in many different forms, and if you look at all the other applications out there with CrystalHD support, none of them really even try to handle it. This inherent challenge is compounded by the fact that the frames don’t seem to get marked correctly when returned from the hardware, so it’s hard to tell the cases apart. I have been able to verify that MPEG2 style interlacing (where the hardware returns each field separately) and H.264 MBAFF (where the hardware returns a field pair as one frame, but with dubious flags) work correctly. H.264 PAFF should work the same as MPEG2 but I haven’t found a sample that isn’t a scary DVB broadcast stream, which introduces all sorts of other complications. There’s also a concept of interlaced content that’s marked as progressive (so you just get a field pair frame) and this should just work but I haven’t found a sample yet.

Other things of note

If you’re trying to play very high bandwidth files, make sure your I/O path is actually fast enough to move the data (so don’t expect Bluray content to work over an 802.11g link). And if your file is encoded with features that aren’t supported by the hardware, then all bets are off; the most common example of this would be an H.264 file where the encoder decided to use the highest possible settings – such as 16 b-frames on 1080p content. The CrystalHD appears to try its best to play these files but you’ll see glitches (eg: In the 16 b-frame case, it will just silently fail to resolve references to the additional b-frames)

Performance

As all codec work is done in hardware, the CPU utilization is purely based on the video resolution – almost all the time is spent copying frames back and forth. In my very unscientific tests, my old 2.2GHz Core 2 Duo laptop can play 1080p content at 25% of a core compared to 70-100% for software decoding. Also note that the X server (and window manager if you use a composited desktop) will burn measurable amounts of CPU time to display the frames. It’s supposed to be possible to do 1080p playback on a single-core Atom, but I’m not in a position the test that. Nevertheless, the benefits are clear.

Getting the code

First off, you’ll need the latest driver and userspace library from Jarod’s git tree.

Then you should grab my patched Mplayer and FFmpeg trees from github, and then construct a full Mplayer tree on disk. That means:

  • mplayer/: My mplayer tree
  • mplayer/ffmpeg/: My ffmpeg tree
  • mplayer/ffmpeg/libswcale/: From mplayerhq
  • mplayer/libdvdread4/: From svn at svn://svn.mplayerhq.hu/dvdnav/trunk/libdvdread/src
  • mplayer/libdvdnav/: From svn at //svn.mplayerhq.hu/dvdnav/trunk/libdvdnav/src

Then you should be able to build Mplayer as normal. CrystalHD support should be auto-detected and will be preferred at playback to the software decoder. In theory, mencoder should also work correctly but I haven’t tried it.

Enjoy!

{ 25 } Comments

  1. Brian Hall | 1st January 2011 at 13:04 | Permalink

    This is good work, thanks! A few questions:

    1) Is your mplayer tree using the multithreaded branch of mplayer? I’ve been using mplayer-mt-git quite successfully on my dual core, dual threaded Atom (Intel D510MO board).

    2) Does mplayer+crystalhd work on 64-bit Linux or just 32-bit?

    3) Any chance your patches will make it into either mainstream mplayer or mplayer-mt soon-ish?

  2. Philip Langdale | 1st January 2011 at 13:45 | Permalink

    Hi Brian,

    1) There isn’t really an mplayer-mt as far as I can tell, just mplayer using ffmpeg-mt. I have not been working against that because it’s not upstream ffmpeg but my patches should apply against it. Mind you, with ffmpeg-mt, the design of the decoder really should be changed to take advantage of the multi-threading. Ideally, you’d want a separate input and output thread – and this is what happens with XBMC and Xine, for example. ffmpeg-mt appears to fork per-frame threads, which is not exactly ideal because you’d end up with a single thread pulling frames out and then having to work out which frame-thread the frame belonged to and passing it over so that thread could return; I think you’d end up having to serialize in so many places that you’d lose any benefits.

    2) Yes; I only work with 64bit.

    3) Soon-ish? No. I can’t see the ffmpeg patches going in until you can actually use crystalhd with ffmpeg itself or ffplay, and that won’t be true until the decoder lag work is done. On top of that, they may be unwilling to accept it until the interlacing support is complete, so there’s likely a fair amount of work left.

  3. Miroslav Talasek | 2nd January 2011 at 00:03 | Permalink

    so i still can see error but video plays correctly
    Playing /home/miro/HDTest/1080pTest.mkv.
    libavformat file format detected.
    Running DIL (3.21.0) Version
    DtsDeviceOpen: Opening HW in mode 0
    Enable single threaded mode
    Scaling command param 0×0,ctx_scal:0×0
    DtsAllocIoctlData Error
    [matroska,webm @ 0xa425620] Estimating duration from bitrate, this may be inaccurate
    [lavf] stream 0: video (h264_crystalhd), -vid 0
    [lavf] stream 1: audio (aac), -aid 0
    VIDEO: [H264] 1920×816 0bpp 23.976 fps 0.0 kbps ( 0.0 kbyte/s)
    ==========================================================================
    Opening video decoder: [ffmpeg] FFmpeg’s libavcodec codec family
    Running DIL (3.21.0) Version
    DtsDeviceOpen: Opening HW in mode 0
    Enable single threaded mode
    Scaling command param 0×0,ctx_scal:0×0
    Selected video codec: [h264crystalhd] vfm: ffmpeg (CrystalHD H.264)

    every thing seems to be ok

  4. Miroslav Talasek | 2nd January 2011 at 03:11 | Permalink

    But i see shuttering in 1080p playback but it is problem of nevest driver and library because this problem appear in gstreamer and xbmc too . but only with recent drivers and library

  5. Ernst | 2nd January 2011 at 10:51 | Permalink

    Any example of netbooks with the 70015 chip?

  6. Philip Langdale | 2nd January 2011 at 11:11 | Permalink

    The DtsAllocIoctlData error is normal. Mplayer decodes a few frames to check the data and then stops – this is what causes that message.

  7. Philip Langdale | 2nd January 2011 at 11:12 | Permalink

    Do you know which driver/library changeset this didn’t happen with?

  8. Philip Langdale | 2nd January 2011 at 11:12 | Permalink

    The Dell Mini 10 is one example.

  9. Miroslav Talasek | 2nd January 2011 at 13:00 | Permalink

    2Philip:
    Hi
    i dont known exactly but i know that this version
    http://www.broadcom.com/docs/support/crystalhd/crystalhd_linux_20100703.zip
    work properly yet (without any shuttering)

    2Ernst:
    My example is my modified Asus EEE 901 (with soldered second minipci-e) with halfsize 70015 chip ;)

  10. Miroslav Talasek | 3rd January 2011 at 11:37 | Permalink

    I try older firmware from http://www.broadcom.com/docs/support/crystalhd/crystalhd_linux_20100703.zip and it seems better but after seeking still shuttering

  11. Holla | 18th January 2011 at 02:18 | Permalink

    Hi Philip, great work ! thanks.

    I am putting together a mythtv frontend/backend using Intel Atom board D510MO and PVR350 card. Only output is through the tv-out of this card (display resolution 720×576).
    Entire root file system and the kernel is built using the OpenEmbedded framework.

    Recently I added Broadcom 70015 and managed to bring your mplayer/ffmpeg work to this setup.

    > root@d510mo:~# mplayer -vo xv -aspect 16:9 big_buck_bunny_1080p_h264.mov

    MPlayer UNKNOWN-4.5.0 (C) 2000-2011 MPlayer Team
    Playing big_buck_bunny_1080p_h264.mov.
    libavformat file format detected.
    Running DIL (3.21.0) Version
    DtsDeviceOpen: Opening HW in mode 0
    Enable single threaded mode
    Scaling command param 0×0,ctx_scal:0×0
    [h264_crystalhd @ 0xa207990] CrystalHD: No frames ready. Returning
    [h264_crystalhd @ 0xa207990] CrystalHD: No frames ready. Returning
    [h264_crystalhd @ 0xa207990] CrystalHD: No frames ready. Returning
    [h264_crystalhd @ 0xa207990] CrystalHD: No frames ready. Returning
    [h264_crystalhd @ 0xa207990] CrystalHD: No frames ready. Returning
    [h264_crystalhd @ 0xa207990] CrystalHD: No frames ready. Returning
    [h264_crystalhd @ 0xa207990] CrystalHD: No frames ready. Returning
    [h264_crystalhd @ 0xa207990] CrystalHD: No frames ready. Returning
    DtsAllocIoctlData Error
    [lavf] stream 0: video (h264_crystalhd), -vid 0
    [lavf] stream 2: audio (aac), -aid 0, -alang eng
    VIDEO: [H264] 1920×1080 24bpp 24.000 fps 9282.6 kbps (1133.1 kbyte/s)
    Clip info:
    major_brand: qt
    minor_version: 537199360
    compatible_brands: qt
    creation_time: 2008-05-27 18:40:35
    Load subtitles in /mnt/store/oedstore/video/
    Error opening/initializing the selected video_out (-vo) device.
    ==========================================================================
    Forced audio codec: mad
    Opening audio decoder: [ffmpeg] FFmpeg/libavcodec audio decoders
    AUDIO: 48000 Hz, 6 ch, s16le, 437.6 kbit/9.50% (ratio: 54700->576000)
    Selected audio codec: [ffaac] afm: ffmpeg (FFmpeg AAC (MPEG-2/MPEG-4 Audio))
    ==========================================================================
    AO: [alsa] 48000Hz 2ch floatle (4 bytes per sample)
    Video: no video
    Starting playback…
    A: 7.4 (07.3) of 596.5 (09:56.4) 2.9%

    As you can see , CrystalHD decoder is working but there is a problem in using the xv video out. Its resolution is 720×576 and the decoder output is at 1920×1080.

    Any way to bridge this ?

    If -vo fbdev is used , I get to see a slideshow…

  12. Philip Langdale | 18th January 2011 at 09:56 | Permalink

    A normal graphics card’s Xv support will downscale video to fit the screen resolution. However, the PVR350 is rather limited and can’t handle input that’s over 720×576. You should be able to use ‘-vf scale’ to insert a scale filter to the rescaling in software before it goes to Xv. The mplayer man page explains how to do this. The CrystalHD is actually capable of doing scaling in hardware so that the decoded frames are the right size but I don’t know how to expose that capability in a way that ffmpeg/mplayer can use – you’d have to edit the patch to turn the hw scaling on and hard-code the output resolution.

  13. Bob Halloran | 18th January 2011 at 11:37 | Permalink

    Anyone know of a PCI/PCIe CrystalHD card available, and whether the mods above would work with it? I’m aware of the mini-PCIe hardware available, and PCIe carrier cards for desktops, but unsure whether the CrystalHD hardware would be properly detected with such an arrangement. Thanks in advance.

  14. Philip Langdale | 18th January 2011 at 12:56 | Permalink

    There’s no native desktop card available, but the adapters will work fine – they’re completely passive so there’s nothing on them to introduce incompatibilities.

  15. Bob Halloran | 18th January 2011 at 13:05 | Permalink

    Much thanks for the update; I thought as much about using the PCIe carriers, but wanted to confirm my suspicions.

  16. Daniel | 20th January 2011 at 12:13 | Permalink

    silly question probably, but when running ./configure of mplayer should “Checking for CrystalHD … ” read “yes” ?

    I followed your instructions but i keep getting “Checking for CrystalHD … no”
    Where am i going wrong?

  17. Philip Langdale | 20th January 2011 at 18:59 | Permalink

    Did you actually install libcrystalhd as discussed in the post?

  18. Daniel | 20th January 2011 at 23:42 | Permalink

    You are absolutely correct, i did install the driver but forgot to install the lib. Thanks for pointing that out to me.

  19. Tomas | 5th February 2011 at 03:07 | Permalink

    Is this possible to compile for windows to? looks like the libcrystalhd only exists for linux?

  20. Philip Langdale | 5th February 2011 at 08:11 | Permalink

    In theory, but it’ll require a fair amount of hacker ingenuity. libcrystalhd is cross platform and ships as part of the windows driver package, IIRC, but there are no headers or import lib. There’s a reasonable chance that the linux headers would work – they contain comments that show their windows origins but you’d have to glue everything together yourself.

    Ultimately, on windows, you’re better off using the dxva support in ffmpeg; the broadcom drivers will work with that.

  21. Yionel | 22nd February 2011 at 14:54 | Permalink

    Hi, (I’m French so sorry for my speak)

    I bought a crystal HD on my netbook D150 Acer on Ubuntu 10.10, I install the drivers, but now, I don’t understand (I’m a noob) how to install the patch to read HD movies with mplayer.
    Can you explain more ?

    Thanks a lot !!

  22. Maic Striepe | 24th March 2011 at 14:18 | Permalink

    Hi!
    First: Thanx for your work on this.
    Unfortunatly I could not get this to work is the procedure mentioned above still valid? For example mplayer/ffmpeg/libswcale/ links seems not correct anymore. Is in ffmpeg now if I understand correct. I got it compiled but mplayer still does not use the crystalhd-card :-/

  23. Philip Langdale | 24th March 2011 at 17:59 | Permalink

    The code is now actually in upstream ffmpeg and mplayer. I need to write a new blog post about that. The instructions are still correct, except that libswscale is bundled into ffmpeg now. But as it’s all upstream, just follow the upstream instructions to get and build the code.

  24. MrTodManning | 29th March 2011 at 19:31 | Permalink

    Thanks Philip!

    This is actually great news for me as I’m really having bad luck with XBMC. I have a BCM70015 (I had 70012 before) and I want to switch to Linux full time. Before, I had to boot to windows (dual) just to watch HD videos since Linux support was not that great yet. Unfortunately, I’m a new linux user and I don’t know what to do with those folders after downloading.

    I’m just wondering if you, or someone else, could post step-by-step easy guide on how to compile and install this properly. I’m sure some new linux users would be interested on this as well.

    Thank you very much!

  25. Matt | 27th August 2011 at 18:51 | Permalink

    What is the status on crystalhd support in ffmpeg? Everytime I ./configure the source tree does not recognize my 70015 and ALSO the build errors out at /libavcodec/libcrystalhd

    Since the post before me hasn’t been answered I don’t expect a response but it would be great to hear some feedback……

{ 9 } Trackbacks

  1. [...] Philip Langdale Posted by Bez kategorii Subscribe to RSS feed [...]

  2. [...] медиаплеера MPlayer и мультимедиа пакета FFmpeg подготовлен набор патчей с реализацией поддержки декодирования [...]

  3. [...] медиаплеера MPlayer и мультимедиа пакета FFmpeg подготовлен набор патчей с реализацией поддержки декодирования [...]

  4. [...] This post was mentioned on Twitter by Răzvan Sandu, Zuissi. Zuissi said: Planet Gnome: Philip Langdale: Broadcom CrystalHD Decoder support for FFmpeg and MPlayer: At the end of last yea… http://bit.ly/e2XzTj [...]

  5. MPlayer wspiera CrystalHD | /home/artek | 15th April 2011 at 11:45 | Permalink

    [...] http://intr.overt.org/blog/?p=117 AKPC_IDS += "858,"; Ten wpis umieszczono w kategorii Miniposty i otagowano jako crystalhd, mplayer. Możesz dodać go do zakładek permalink. Dodaj komentarz lub dodaj odpowiedź (trackback): Trackback URL. « mini-arduino [...]

  6. [...] Więcej znajdziemy na blogu autora. [...]

  7. [...] Member I have built mplayer from git tree as per here http://intr.overt.org/blog/?p=117 Is still compiling I will let you know when is done what the results are Posted 1 hour ago [...]

  8. short guide to pvr « Stm Labs Forums | 28th August 2011 at 22:09 | Permalink

    [...] http://intr.overt.org/blog/?p=117 [...]

  9. [...] znajdziemy na blogu autora. Tweet var wykop_url=location.href; var wykop_title=encodeURIComponent(document.title); var [...]