Enabling accelerated video decoding in Firefox on Ubuntu 21.04

vchernin · April 28, 2021, 9:40pm

In this past year there has been work to make Firefox hardware decode video on Linux. Ubuntu 21.04’s switch to wayland luckily makes this feature more accessible and easier to enable.

Benefits:
Hardware video decoding in Firefox greatly reduces CPU usage when watching video. This saves power, improves thermals and reduces fan noise, all while ensuring computer resources are more free to do other things while watching video.

While the feature is not perfect, I have been reliably using it for several months now on various distros. For me this feature is the difference between 4k web video being playable or not at all.

Here are steps for Ubuntu 21.04, for devices with Intel and AMD graphics (Nvidia is problematic). Older Ubuntu versions and other distros may need additional steps as mentioned below.

Steps:

Ensure all media codecs and libraries are installed with sudo apt install ubuntu-restricted-extras. This may not be necessary, but I found it necessary on a fresh 21.04 install (I didn’t check off the box to third party drivers during installation).
Navigate to about:config in Firefox and set the following flags:

media.ffmpeg.vaapi.enabled to true. Enables the feature.

media.ffvpx.enabled to false. Disables the internal software decoders for VP8/VP9, which currently overrule hardware accelerated video.

media.rdd-vpx.enabled to false. Needed due to Firefox trying to put some video decoding in a remote video process. This currently breaks accelerated VP8/VP9 video decoding.

media.navigator.mediadatadecoder_vpx_enabled to true. Enables hardware decoding for WebRTC, which is used on web based video conferencing apps.

Restart Firefox
On Intel hardware, test the feature by running the following :
sudo apt install intel-gpu-tools
sudo intel_gpu_top
If the video bar is above 0% the feature is working.
As a general test you can compare cpu usage when this feature is enabled vs disabled.

If the feature isn’t working navigate to about:support and ensure you see the following values. Note on relatively modern Intel and AMD hardware these are usually enabled by default.
Compositing: WebRender
Window Protocol: wayland

WebRender is Firefox’s hardware accelerated rendering, which is needed for video decoding. You may also have luck with Firefox’s legacy OpenGL renderer. This feature is also possible to enable on x11 as mentioned below (I’d recommend wayland since I have found x11 to perform much worse with this feature).

Caveats:
A not linux-specific issue is that YouTube will try to play video codecs that are not hardware accelerated by your device. This extension lets you control what codecs YouTube uses. To confirm this in YouTube right click the video and click ‘stats for nerds’ to ensure the video codec used can be accelerated by your gpu. For Intel gpu video codec support see this matrix.

The feature does not currently seem to work on Nvidia hardware. This is likely since Nvidia does not natively support VA-API and dmabuf which this Firefox feature uses.

YouTube VR videos do not seem to work when this feature is enabled.

Sometimes I notice the video output turns solid green after it has been paused. Since this immediately goes away after playback is resumed I don’t consider it a major issue.

Devices with multiple gpus, such as a device with an integrated Intel gpu and a dedicated Nvidia gpu may not work correctly with this feature.

If you are on Ubuntu 20.04 or non wayland sessions you can still enable this feature, however, you may need to complete a extra steps (in addition to the above instructions).
To enable the feature on xorg append:
export MOZ_X11_EGL=1 to /etc/environment

To enable the feature on wayland append (not needed on 21.04):
export MOZ_ENABLE_WAYLAND=1 to /etc/environment

Thoughts:
Even with some issues that come with this feature (like YouTube VR video being broken) there are massive performance upsides with higher resolution video. Hopefully, if there is not too much breakage this feature could be enabled by default in future Ubuntu releases.

This info is mostly sourced from these pages as well as my own testing:
Comprehensive Arch Wiki page with more information
The meta Firefox bug for this issue
Blog post when the feature first landed

emcp · September 19, 2021, 11:12am

I just tried this on 20.04 LTS with an NVIDIA GPU… seems not to break my video but doesn’t maybe do much else either…

IMO until it works with NVIDIA it would be foolish to put on by default

EDIT:

I just tried pushing this thing with an 8K youtube video… seems to be slightly slightly smoother… I don’t want to fiddle back and forth but… maybe you have something stable since the time you posted and it works ??

vchernin · September 22, 2021, 4:13pm

As mentioned in the arch wiki page, for this feature to work at all you’d at least need Nvidia 470 drivers for it to maybe work. I have no confirmation of whether this feature works for anyone on Nvidia. If someone wants to test it please do.

Enabling Wayland mode and/or WebRender is generally much better for smoothness in Firefox anyways, regardless of whether VAAPI is working.

mrw · October 10, 2021, 4:35pm

Yep, these steps don’t work on my system with both Intel HD Graphics 620 (i7-7500U) and NVIDIA GeForce GTX 940MX. Tried these settings on Ubuntu 21.04 with Nvidia Prime set to use Power Saving GPU (intel).
I can see the render bar in intel_gpu_top going up while playing 4k HDR video, but the Video bar is always at 0%. The video is super choppy and unplayable.

If anyone has had any luck with Lenovo Yoga 710 15I-KB, please let me know.

vanvugt · October 13, 2021, 1:36am

@mrw,

You should check the video format you are trying to decode on your i7-7500U (Kaby Lake) against the matrix of supported codecs. FYI, if it’s YouTube then I think they have moved to AV1 for “4k HDR” so you would at least need an 11th gen Intel CPU

If you still think it should be decoding in hardware then you should report a browser bug.

mrw · October 13, 2021, 7:57am

It is a YouTube video - https://www.youtube.com/watch?v=LXb3EKWsInQ

The same video plays beautifully at full 4k resolution on Windows 10. Task Manager clearly shows intel gpu being used and there are virtually no frame drops.

I don’t think it’s specific to browser since I have the same issue with Chromium on Ubuntu 20.04.

vanvugt · October 13, 2021, 8:22am

@mrw,

On my Linux desktop, YouTube appears to be offering that video in VP9 format. So yes your Intel i7-7500U can decode it efficiently in theory . In practice however that requires the web browser to support VA-API to access the nice hardware decoding features. Therefore high CPU and “0% Video” reported by intel_gpu_top indicates a problem with the browser; please log a bug.

vanvugt · October 13, 2021, 8:48am

Nevermind, we already have bug 1424201 open for this. I’m not sure why upstream Firefox has the issue closed because I can’t get it working either…

P.S. The Chrome developers seem to have also declared the issue won’t be fixed.

mrw · October 13, 2021, 2:04pm

Yeah until this is fixed I am sticking to Windows 10.

No point in having a 4k display and not being able to use it.

vchernin · October 13, 2021, 7:43pm

Could you confirm you’ve followed this guide, and that other applications (eg VLC) are able to use VA-API on your system?

Could you try setting media.rdd-process.enabled to false in about:config? (In addition to all the other changes instructed in the guide).

This guide is still necessary (and should be up to date). The main issue blocking this as a default is moving VA-API to the GPU process, which will resolve/avoid some of the sandboxing issues. There is a summary of everything in the Arch Wiki.

mrw · October 13, 2021, 9:26pm

I have followed this guide. Here are my about:config settings:
media.ffmpeg.vaapi.enabled : true
media.ffvpx.enabled : false
media.rdd-vpx.enabled : false
mediadatadecoder_vpx_enabled : false

$ sudo apt install ubuntu-restricted-extras
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
ubuntu-restricted-extras is already the newest version (67).

$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 21.04
Release:	21.04
Codename:	hirsute

about:support values –

Compositing: WebRender (Software)
Window Protocol: wayland

intel_gpu_top output while playing the video

      ENGINE      BUSY                                      MI_SEMA MI_WAIT
 Render/3D/0   16.50% |█████▊                             |      0%      0%
   Blitter/0    0.00% |                                   |      0%      0%
     Video/0    0.00% |                                   |      0%      0%

VideoEnhance/0 0.00% | | 0% 0%

^C

I don’t have a 4k video to test in VLC.

No improvement after setting media.rdd-process.enabled to false

      ENGINE      BUSY                                      MI_SEMA MI_WAIT
 Render/3D/0   14.35% |█████                              |      0%      0%
   Blitter/0    0.00% |                                   |      0%      0%
     Video/0    0.00% |                                   |      0%      0%

VideoEnhance/0 0.00% | | 0% 0%

^C

vchernin · October 13, 2021, 9:39pm

WebRender (Software)

That’s your problem. Software WebRender is still software rendering. Hardware rendering (WebRender) is a prerequisite in order to use VA-API in Firefox. Try setting gfx.webrender.all to true and restart Firefox.

The arch wiki explicitly mentions this, unfortunately this guide did not. I would edit the guide but the edit button no longer shows up, probably because the post is old?

mrw · October 13, 2021, 10:27pm

I set gfx.webrender.all to true and now it shows:
Compositing WebRender

But the choppy video and frame dropping issue still persists.

      ENGINE      BUSY                                      MI_SEMA MI_WAIT
 Render/3D/0   16.49% |█████▊                             |      0%      0%
   Blitter/0    0.00% |                                   |      0%      0%
     Video/0    0.00% |                                   |      0%      0%

VideoEnhance/0 0.00% | | 0% 0%

^C

vchernin · October 13, 2021, 10:52pm

Hm, then it’s probably something else then.

As one more sanity check can you verify VP9 or H264 is shown in YouTube when right clicking the video > stats for nerds.

If that is VP9 or H264 then I’m out of simple solutions. Everything you’ve set looks correct.

I’d recommend checking the output of things like vainfo and running Firefox with MOZ_LOG="PlatformDecoderModule:5" firefox. Anything mentioning VA-API is relevant.

mrw · October 13, 2021, 11:09pm

It is vp09, i don’t see H264 here for that video. As you can see 1000s of frames are dropped as the video plays.

$ vainfo
libva info: VA-API version 1.10.0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_10
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.10 (libva 2.10.0)
vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 21.1.1 ()
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            :	VAEntrypointVLD
      VAProfileMPEG2Main              :	VAEntrypointVLD
      VAProfileH264Main               :	VAEntrypointVLD
      VAProfileH264Main               :	VAEntrypointEncSliceLP
      VAProfileH264High               :	VAEntrypointVLD
      VAProfileH264High               :	VAEntrypointEncSliceLP
      VAProfileJPEGBaseline           :	VAEntrypointVLD
      VAProfileJPEGBaseline           :	VAEntrypointEncPicture
      VAProfileH264ConstrainedBaseline:	VAEntrypointVLD
      VAProfileH264ConstrainedBaseline:	VAEntrypointEncSliceLP
      VAProfileVP8Version0_3          :	VAEntrypointVLD
      VAProfileHEVCMain               :	VAEntrypointVLD
      VAProfileHEVCMain10             :	VAEntrypointVLD
      VAProfileVP9Profile0            :	VAEntrypointVLD
      VAProfileVP9Profile2            :	VAEntrypointVLD

Running with

$ MOZ_LOG="PlatformDecoderModule:5" firefox

but still the same issue.

mrw · October 13, 2021, 11:17pm

Something interesting I found in the logs when running with MOZ_LOG

[Child 6908: Main Thread]: D/PlatformDecoderModule VA-API FFmpeg is disabled by platform
[Child 6908: Main Thread]: D/PlatformDecoderModule VA-API FFmpeg is disabled by platform

But I did enable ffmpeg vaapi and about:support also confirms this:

media.ffmpeg.vaapi.enabled true

vchernin · October 13, 2021, 11:18pm

This variable doesn’t change how Firefox works, it just logs what it’s doing. Maybe there’s some clues there.

Aside from maybe a new Firefox profile being useful I don’t see anything obviously wrong here. It’s probably a Firefox bug somewhere.

vchernin · October 13, 2021, 11:19pm

Yes that is probably a problem. Can you try in a fresh Firefox profile and try the guide?

mrw · October 13, 2021, 11:21pm

Sorry what do you mean fresh profile? Should i uninstall and reinstall firefox?

mrw · October 13, 2021, 11:22pm

media.hardware-video-decoding.force-enabled is also true