At the end of last year, Broadcom released open-source drivers and a library for their CrystalHD hardware video decoder; You can read the details about that at Jarod Wilson’s blog if you’re interested.
The hardware is particularly attractive because it’s low cost and can be added to any system, regardless of the GPU it uses. It provides MPEG1/2, H.264 and VC-1 decode capabilities in all hardware versions, and the latest 70015 part also adds MPEG4 Part 2 / DivX / XviD support – and, if you care about such things, it does so in a way that means all the infamous patent issues are handled in hardware.
Once the drivers and library were released, we started to see plenty of application support emerge, with XBMC, Xine and MythTV drivers under development early on, and a gstreamer plugin provided by Broadcom with the library. In the last couple of months, a VLC patch has been proposed. But, for all this, there was no movement on FFmpeg or Mplayer; the first being the most widely used codec library around and the later, one of the most widely used media players (and obviously a consumer of FFmpeg). I bought myself a 70012 and later a 70015 with the intention of playing around with them, but when I got some free time last month, I started working on FFmpeg and Mplayer support.
After some experimentation, I did the initial implementation as a native Mplayer decoder, which helped remove FFmpeg as a variable while I tried to get things working correctly, and after some effort, I came up with something that worked pretty well for progressive content. and partially worked for some interlaced content. In my initial discussion on the mplayer mailing list, it became clear that it needed to be an FFmpeg decoder to be maintainable for the long term. So I went back and converted it, which was relatively straight-forward as the APIs are very similar. I’ve now started getting review feedback on the FFmpeg list, and I expect it will be quite a while before it gets in, assuming it ever does, but the code is definitely usable enough to publicise more widely.
What Works
- MPlayer playback
- 70015 Hardware: This is the newest part, with extra codec support
- All the officially supported content types except DivX 3.11
- Progressive Content
- Interlaced MPEG2 and H.264 MBAFF Content
You’ll note that I said MPlayer playback works but nothing about FFmpeg even though it’s an FFmpeg decoder. The issue here is that the hardware is pipelined, so it is normal for many frames to be in flight at one time – perhaps as many as 20 under normal conditions, and over a hundred if things start going wrong… This means that the application has to have a concept of lag in the decoding process. Now, while Mplayer uses FFmpeg’s codecs, it handles frame timing and av sync in a completely different way from the actual FFmpeg transcoding application, and it happens to do it in a way that can cope with the hardware behaviour (although I had to increase the number of frames in flight that Mplayer can handle). FFmpeg on the other hand assumes that when it submits a frame for decoding, the frame it gets back is the same one. From the discussion, it sounds like I’ll need to add the same mechanism that Mplayer uses to FFmpeg. So, for now, it’s an FFmpeg decoder that doesn’t work with FFmpeg 
The 70012 is the previous generation of hardware and it has some significant differences from the 70015. Beyond the reduced codec support, it has much tighter requirements with respect to keeping the pipeline happily fed, and I haven’t had a chance to investigate what this really means yet. For now, it will kind-of work, but expect something to go wrong with the pipeline and either get input overflow or output underflow pretty quickly.
Interlaced content is tricky because it can appear in many different forms, and if you look at all the other applications out there with CrystalHD support, none of them really even try to handle it. This inherent challenge is compounded by the fact that the frames don’t seem to get marked correctly when returned from the hardware, so it’s hard to tell the cases apart. I have been able to verify that MPEG2 style interlacing (where the hardware returns each field separately) and H.264 MBAFF (where the hardware returns a field pair as one frame, but with dubious flags) work correctly. H.264 PAFF should work the same as MPEG2 but I haven’t found a sample that isn’t a scary DVB broadcast stream, which introduces all sorts of other complications. There’s also a concept of interlaced content that’s marked as progressive (so you just get a field pair frame) and this should just work but I haven’t found a sample yet.
Other things of note
If you’re trying to play very high bandwidth files, make sure your I/O path is actually fast enough to move the data (so don’t expect Bluray content to work over an 802.11g link). And if your file is encoded with features that aren’t supported by the hardware, then all bets are off; the most common example of this would be an H.264 file where the encoder decided to use the highest possible settings – such as 16 b-frames on 1080p content. The CrystalHD appears to try its best to play these files but you’ll see glitches (eg: In the 16 b-frame case, it will just silently fail to resolve references to the additional b-frames)
Performance
As all codec work is done in hardware, the CPU utilization is purely based on the video resolution – almost all the time is spent copying frames back and forth. In my very unscientific tests, my old 2.2GHz Core 2 Duo laptop can play 1080p content at 25% of a core compared to 70-100% for software decoding. Also note that the X server (and window manager if you use a composited desktop) will burn measurable amounts of CPU time to display the frames. It’s supposed to be possible to do 1080p playback on a single-core Atom, but I’m not in a position the test that. Nevertheless, the benefits are clear.
Getting the code
First off, you’ll need the latest driver and userspace library from Jarod’s git tree.
Then you should grab my patched Mplayer and FFmpeg trees from github, and then construct a full Mplayer tree on disk. That means:
- mplayer/: My mplayer tree
- mplayer/ffmpeg/: My ffmpeg tree
- mplayer/ffmpeg/libswcale/: From mplayerhq
- mplayer/libdvdread4/: From svn at svn://svn.mplayerhq.hu/dvdnav/trunk/libdvdread/src
- mplayer/libdvdnav/: From svn at //svn.mplayerhq.hu/dvdnav/trunk/libdvdnav/src
Then you should be able to build Mplayer as normal. CrystalHD support should be auto-detected and will be preferred at playback to the software decoder. In theory, mencoder should also work correctly but I haven’t tried it.
Enjoy!
{ 25 } Comments
This is good work, thanks! A few questions:
1) Is your mplayer tree using the multithreaded branch of mplayer? I’ve been using mplayer-mt-git quite successfully on my dual core, dual threaded Atom (Intel D510MO board).
2) Does mplayer+crystalhd work on 64-bit Linux or just 32-bit?
3) Any chance your patches will make it into either mainstream mplayer or mplayer-mt soon-ish?
Hi Brian,
1) There isn’t really an mplayer-mt as far as I can tell, just mplayer using ffmpeg-mt. I have not been working against that because it’s not upstream ffmpeg but my patches should apply against it. Mind you, with ffmpeg-mt, the design of the decoder really should be changed to take advantage of the multi-threading. Ideally, you’d want a separate input and output thread – and this is what happens with XBMC and Xine, for example. ffmpeg-mt appears to fork per-frame threads, which is not exactly ideal because you’d end up with a single thread pulling frames out and then having to work out which frame-thread the frame belonged to and passing it over so that thread could return; I think you’d end up having to serialize in so many places that you’d lose any benefits.
2) Yes; I only work with 64bit.
3) Soon-ish? No. I can’t see the ffmpeg patches going in until you can actually use crystalhd with ffmpeg itself or ffplay, and that won’t be true until the decoder lag work is done. On top of that, they may be unwilling to accept it until the interlacing support is complete, so there’s likely a fair amount of work left.
so i still can see error but video plays correctly
Playing /home/miro/HDTest/1080pTest.mkv.
libavformat file format detected.
Running DIL (3.21.0) Version
DtsDeviceOpen: Opening HW in mode 0
Enable single threaded mode
Scaling command param 0×0,ctx_scal:0×0
DtsAllocIoctlData Error
[matroska,webm @ 0xa425620] Estimating duration from bitrate, this may be inaccurate
[lavf] stream 0: video (h264_crystalhd), -vid 0
[lavf] stream 1: audio (aac), -aid 0
VIDEO: [H264] 1920×816 0bpp 23.976 fps 0.0 kbps ( 0.0 kbyte/s)
==========================================================================
Opening video decoder: [ffmpeg] FFmpeg’s libavcodec codec family
Running DIL (3.21.0) Version
DtsDeviceOpen: Opening HW in mode 0
Enable single threaded mode
Scaling command param 0×0,ctx_scal:0×0
Selected video codec: [h264crystalhd] vfm: ffmpeg (CrystalHD H.264)
every thing seems to be ok
But i see shuttering in 1080p playback but it is problem of nevest driver and library because this problem appear in gstreamer and xbmc too . but only with recent drivers and library
Any example of netbooks with the 70015 chip?
The DtsAllocIoctlData error is normal. Mplayer decodes a few frames to check the data and then stops – this is what causes that message.
Do you know which driver/library changeset this didn’t happen with?
The Dell Mini 10 is one example.
2Philip:
Hi
i dont known exactly but i know that this version
http://www.broadcom.com/docs/support/crystalhd/crystalhd_linux_20100703.zip
work properly yet (without any shuttering)
2Ernst:
My example is my modified Asus EEE 901 (with soldered second minipci-e) with halfsize 70015 chip
I try older firmware from http://www.broadcom.com/docs/support/crystalhd/crystalhd_linux_20100703.zip and it seems better but after seeking still shuttering
Hi Philip, great work ! thanks.
I am putting together a mythtv frontend/backend using Intel Atom board D510MO and PVR350 card. Only output is through the tv-out of this card (display resolution 720×576).
Entire root file system and the kernel is built using the OpenEmbedded framework.
Recently I added Broadcom 70015 and managed to bring your mplayer/ffmpeg work to this setup.
> root@d510mo:~# mplayer -vo xv -aspect 16:9 big_buck_bunny_1080p_h264.mov
MPlayer UNKNOWN-4.5.0 (C) 2000-2011 MPlayer Team
Playing big_buck_bunny_1080p_h264.mov.
libavformat file format detected.
Running DIL (3.21.0) Version
DtsDeviceOpen: Opening HW in mode 0
Enable single threaded mode
Scaling command param 0×0,ctx_scal:0×0
[h264_crystalhd @ 0xa207990] CrystalHD: No frames ready. Returning
[h264_crystalhd @ 0xa207990] CrystalHD: No frames ready. Returning
[h264_crystalhd @ 0xa207990] CrystalHD: No frames ready. Returning
[h264_crystalhd @ 0xa207990] CrystalHD: No frames ready. Returning
[h264_crystalhd @ 0xa207990] CrystalHD: No frames ready. Returning
[h264_crystalhd @ 0xa207990] CrystalHD: No frames ready. Returning
[h264_crystalhd @ 0xa207990] CrystalHD: No frames ready. Returning
[h264_crystalhd @ 0xa207990] CrystalHD: No frames ready. Returning
DtsAllocIoctlData Error
[lavf] stream 0: video (h264_crystalhd), -vid 0
[lavf] stream 2: audio (aac), -aid 0, -alang eng
VIDEO: [H264] 1920×1080 24bpp 24.000 fps 9282.6 kbps (1133.1 kbyte/s)
Clip info:
major_brand: qt
minor_version: 537199360
compatible_brands: qt
creation_time: 2008-05-27 18:40:35
Load subtitles in /mnt/store/oedstore/video/
Error opening/initializing the selected video_out (-vo) device.
==========================================================================
Forced audio codec: mad
Opening audio decoder: [ffmpeg] FFmpeg/libavcodec audio decoders
AUDIO: 48000 Hz, 6 ch, s16le, 437.6 kbit/9.50% (ratio: 54700->576000)
Selected audio codec: [ffaac] afm: ffmpeg (FFmpeg AAC (MPEG-2/MPEG-4 Audio))
==========================================================================
AO: [alsa] 48000Hz 2ch floatle (4 bytes per sample)
Video: no video
Starting playback…
A: 7.4 (07.3) of 596.5 (09:56.4) 2.9%
As you can see , CrystalHD decoder is working but there is a problem in using the xv video out. Its resolution is 720×576 and the decoder output is at 1920×1080.
Any way to bridge this ?
If -vo fbdev is used , I get to see a slideshow…
A normal graphics card’s Xv support will downscale video to fit the screen resolution. However, the PVR350 is rather limited and can’t handle input that’s over 720×576. You should be able to use ‘-vf scale’ to insert a scale filter to the rescaling in software before it goes to Xv. The mplayer man page explains how to do this. The CrystalHD is actually capable of doing scaling in hardware so that the decoded frames are the right size but I don’t know how to expose that capability in a way that ffmpeg/mplayer can use – you’d have to edit the patch to turn the hw scaling on and hard-code the output resolution.
Anyone know of a PCI/PCIe CrystalHD card available, and whether the mods above would work with it? I’m aware of the mini-PCIe hardware available, and PCIe carrier cards for desktops, but unsure whether the CrystalHD hardware would be properly detected with such an arrangement. Thanks in advance.
There’s no native desktop card available, but the adapters will work fine – they’re completely passive so there’s nothing on them to introduce incompatibilities.
Much thanks for the update; I thought as much about using the PCIe carriers, but wanted to confirm my suspicions.
silly question probably, but when running ./configure of mplayer should “Checking for CrystalHD … ” read “yes” ?
I followed your instructions but i keep getting “Checking for CrystalHD … no”
Where am i going wrong?
Did you actually install libcrystalhd as discussed in the post?
You are absolutely correct, i did install the driver but forgot to install the lib. Thanks for pointing that out to me.
Is this possible to compile for windows to? looks like the libcrystalhd only exists for linux?
In theory, but it’ll require a fair amount of hacker ingenuity. libcrystalhd is cross platform and ships as part of the windows driver package, IIRC, but there are no headers or import lib. There’s a reasonable chance that the linux headers would work – they contain comments that show their windows origins but you’d have to glue everything together yourself.
Ultimately, on windows, you’re better off using the dxva support in ffmpeg; the broadcom drivers will work with that.
Hi, (I’m French so sorry for my speak)
I bought a crystal HD on my netbook D150 Acer on Ubuntu 10.10, I install the drivers, but now, I don’t understand (I’m a noob) how to install the patch to read HD movies with mplayer.
Can you explain more ?
Thanks a lot !!
Hi!
First: Thanx for your work on this.
Unfortunatly I could not get this to work is the procedure mentioned above still valid? For example mplayer/ffmpeg/libswcale/ links seems not correct anymore. Is in ffmpeg now if I understand correct. I got it compiled but mplayer still does not use the crystalhd-card :-/
The code is now actually in upstream ffmpeg and mplayer. I need to write a new blog post about that. The instructions are still correct, except that libswscale is bundled into ffmpeg now. But as it’s all upstream, just follow the upstream instructions to get and build the code.
Thanks Philip!
This is actually great news for me as I’m really having bad luck with XBMC. I have a BCM70015 (I had 70012 before) and I want to switch to Linux full time. Before, I had to boot to windows (dual) just to watch HD videos since Linux support was not that great yet. Unfortunately, I’m a new linux user and I don’t know what to do with those folders after downloading.
I’m just wondering if you, or someone else, could post step-by-step easy guide on how to compile and install this properly. I’m sure some new linux users would be interested on this as well.
Thank you very much!
What is the status on crystalhd support in ffmpeg? Everytime I ./configure the source tree does not recognize my 70015 and ALSO the build errors out at /libavcodec/libcrystalhd
Since the post before me hasn’t been answered I don’t expect a response but it would be great to hear some feedback……
{ 9 } Trackbacks
[...] Philip Langdale Posted by Bez kategorii Subscribe to RSS feed [...]
[...] медиаплеера MPlayer и мультимедиа пакета FFmpeg подготовлен набор патчей с реализацией поддержки декодирования [...]
[...] медиаплеера MPlayer и мультимедиа пакета FFmpeg подготовлен набор патчей с реализацией поддержки декодирования [...]
[...] This post was mentioned on Twitter by Răzvan Sandu, Zuissi. Zuissi said: Planet Gnome: Philip Langdale: Broadcom CrystalHD Decoder support for FFmpeg and MPlayer: At the end of last yea… http://bit.ly/e2XzTj [...]
[...] http://intr.overt.org/blog/?p=117 AKPC_IDS += "858,"; Ten wpis umieszczono w kategorii Miniposty i otagowano jako crystalhd, mplayer. Możesz dodać go do zakładek permalink. Dodaj komentarz lub dodaj odpowiedź (trackback): Trackback URL. « mini-arduino [...]
[...] Więcej znajdziemy na blogu autora. [...]
[...] Member I have built mplayer from git tree as per here http://intr.overt.org/blog/?p=117 Is still compiling I will let you know when is done what the results are Posted 1 hour ago [...]
[...] http://intr.overt.org/blog/?p=117 [...]
[...] znajdziemy na blogu autora. Tweet var wykop_url=location.href; var wykop_title=encodeURIComponent(document.title); var [...]