vaapi does not support VP9 Low Power encoding

[benbaldivia-maumai]

Your environment.

• Version: v0.7.1
• Browser: N/A
• Other Information -

  mediadevices/pkg/codec/vaapi/vp9.go

  Lines 222 to 229 in 20e8c50

  	for i := 0; i < int(numEntrypoints); i++ {
  	if entrypoints[i] == C.VAEntrypointEncSlice {
  	epFound = true
  	}
  	}
  	if !epFound {
  	return nil, errors.New("libva entrypoint not found")
  	}

What did you do?

The codec appears to be hard coded to use only VAEntrypointEncSlice. While my device appears to support encoding viaVAProfileVP9Profile0, it only supports VAEntrypointEncSliceLP. This is not found as the entry point is distinct from VAEntrypointEncSlice and it is hard coded to only look for VAEntrypointEncSlice.

Here is vainfo output from my device:

libva info: VA-API version 1.14.0
libva info: User environment variable requested driver 'iHD'
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_14
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.14 (libva 2.12.0)
vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 22.3.1 ()
vainfo: Supported profile and entrypoints
      VAProfileNone                   :	VAEntrypointVideoProc
      VAProfileNone                   :	VAEntrypointStats
      VAProfileMPEG2Simple            :	VAEntrypointVLD
      VAProfileMPEG2Main              :	VAEntrypointVLD
      VAProfileH264Main               :	VAEntrypointVLD
      VAProfileH264Main               :	VAEntrypointEncSliceLP
      VAProfileH264High               :	VAEntrypointVLD
      VAProfileH264High               :	VAEntrypointEncSliceLP
      VAProfileJPEGBaseline           :	VAEntrypointVLD
      VAProfileJPEGBaseline           :	VAEntrypointEncPicture
      VAProfileH264ConstrainedBaseline:	VAEntrypointVLD
      VAProfileH264ConstrainedBaseline:	VAEntrypointEncSliceLP
      VAProfileHEVCMain               :	VAEntrypointVLD
      VAProfileHEVCMain               :	VAEntrypointEncSliceLP
      VAProfileHEVCMain10             :	VAEntrypointVLD
      VAProfileHEVCMain10             :	VAEntrypointEncSliceLP
      VAProfileVP9Profile0            :	VAEntrypointVLD
      VAProfileVP9Profile0            :	VAEntrypointEncSliceLP
      VAProfileVP9Profile1            :	VAEntrypointVLD
      VAProfileVP9Profile1            :	VAEntrypointEncSliceLP
      VAProfileVP9Profile2            :	VAEntrypointVLD
      VAProfileVP9Profile2            :	VAEntrypointEncSliceLP
      VAProfileVP9Profile3            :	VAEntrypointVLD
      VAProfileVP9Profile3            :	VAEntrypointEncSliceLP
      VAProfileHEVCMain12             :	VAEntrypointVLD
      VAProfileHEVCMain422_10         :	VAEntrypointVLD
      VAProfileHEVCMain422_12         :	VAEntrypointVLD
      VAProfileHEVCMain444            :	VAEntrypointVLD
      VAProfileHEVCMain444            :	VAEntrypointEncSliceLP
      VAProfileHEVCMain444_10         :	VAEntrypointVLD
      VAProfileHEVCMain444_10         :	VAEntrypointEncSliceLP
      VAProfileHEVCMain444_12         :	VAEntrypointVLD
      VAProfileHEVCSccMain            :	VAEntrypointVLD
      VAProfileHEVCSccMain            :	VAEntrypointEncSliceLP
      VAProfileHEVCSccMain10          :	VAEntrypointVLD
      VAProfileHEVCSccMain10          :	VAEntrypointEncSliceLP
      VAProfileHEVCSccMain444         :	VAEntrypointVLD
      VAProfileHEVCSccMain444         :	VAEntrypointEncSliceLP
      VAProfileAV1Profile0            :	VAEntrypointVLD
      VAProfileHEVCSccMain444_10      :	VAEntrypointVLD
      VAProfileHEVCSccMain444_10      :	VAEntrypointEncSliceLP

What did you expect?

When using vaapi with my device with libva installed and support for VAProfileVP9Profile0 detected in the OS for it to work without an error.

What happened?

I got libva entrypoint not found when attempting to use vaapi VP9 encoding.

[3DRX]

@benbaldivia-maumai Thanks for reporting this, I haven’t used vaapi VP9 before, I will try it out and see if I can help to solve your problem.

[3DRX]

Also, can you try using the ffmpeg based encoder in #621 and see if that works for you?

[benbaldivia-maumai]

@3DRX Sounds good. I will see about trying the ffmpeg encoder. As a note, vaapi should work, I expect, if it is used on a device with VAEntrypointEncSlice support. The devices I am using at the moment though, only support VAEntrypointEncSliceLP which is the Low Power implementation of the encoder (as far as I am aware). The code linked in the issue seems to be hard coded to look for only VAEntrypointEncSlice.

When I change the hard coded value to use VAEntrypointEncSliceLP though, I do end up getting an error after a few reads on a vaEndPicture call:

mediadevices/pkg/codec/vaapi/vp9.go

Line 430 in 20e8c50

if s := C.vaEndPicture(

When I look at the error returned it simply says internal encoding error so it was hard for me to troubleshoot the issue from that point. I am not sure if that is because usage of VAEntrypointEncSliceLP needs to be different or if there is just some other issue.

[benbaldivia-maumai]

@3DRX It isn't exactly clear how to set this up. Either way, I was looking at the code and I don't actually see the vp9 codec used. I see vp8 and others. Specifically, I would be looking to use vp9_vaapi. An example usage of this using ffmpeg CLI is:

ffmpeg -vaapi_device /dev/dri/renderD128 -f v4l2 -input_format yuyv422 -video_size 640x480 -framerate 30 -i /dev/video0 -vf 'format=nv12,hwupload' -c:v vp9_vaapi -b:v 200K -f webm temp.webm

So in my case it doesn't look like it would work in it's current form. The reason I was looking to use VP9 was a combination of factors. The first being I want use VP8/9 for license and bitrate/quality reasons. VP8 is fast enough using CPU only but VP9 quality is much better for the same bitrate. The second is that I would like to utilize hardware acceleration because VP9 via CPU encoding results in a significant delay and the later intel CPUs I am working with support VP9 hardware encoding of VP9 (intel docs). The hardware encoding of vp8 via intel iGPU ended after 10th gen and later GPUs have all seemed to support VP9.

[3DRX]

@benbaldivia-maumai Hi, I've updated the ffmpeg based encoder module to support vp9_vaapi. One of the benefits using ffmpeg is that adding new codec support requires minimal code change :)

[benbaldivia-maumai]

@benbaldivia-maumai Hi, I've updated the ffmpeg based encoder module to support vp9_vaapi. One of the benefits using ffmpeg is that adding new codec support requires minimal code change :)

I will definitely try it out. 👍😺

[benbaldivia-maumai]

So, I actually ended up implementing many codecs in a different private package which just uses go-astiav (maybe not too different from the ffmpeg integration PR). However, actually, I had never really had to use the codecs in the way I am which is for real-time encoding and there is expected to be a lot of movement. Unfortunately, the quality that I got when using vaapi was pretty bad and there don't appear to be many options. Supposedly, there are features/functionality within the VP9 encoding itself which are not implemented in (some?) hardware accelerated support for VP9. Personally, I think that is pretty deceptive when I became aware of this as when I see "VP9" support when listing hardware acceleration is not really all that meaningful.

I guess, it depends on the technical requirements. Anyways, VP9 via vaapi is fast, it just doesn't fit my needs.

If I have the time, I will see about releasing the codec package as a separate repository. I actually feel that is probably a better way to provide these codecs where there are specific hardware requirements which can easily be "opt-in". For what it is worth, I also implemented support for all the other VAAPI codecs (which I have hardware support for) including having only codec specific options available for each given codec and option (like only CBR options being available when using CBR).

I will close this issue based on that and maybe just open something to maybe link to the other package if I make it public.