VAAPI profile determination seems to not take hardware limits into account

[Volker Vogelhuber]

I currently have a pipeline where I connect vaapipostproc with 
vaapih264enc. The pipeline runs without problems on an apollo lake 
module but vaapih264enc rejected the caps on a baytrail one. If I 
explicitly define the profile to be high profile it works also on the 
baytrail module. I debugged into the code and it seems like 
encoder->profile_idc is set to encoder->max_profile_idc in 
gstvaapiencoder_h264.c. And max_profile_idc is not taking into account 
the hardware capabilities. Later the profile_idc is used to decide which 
profile should be used in the negotiation phase which is now based on 
the max_profile_idc (set to GST_H264_PROFILE_STEREO_HIGH, for whatever 
reason). But because the baytrail hardware does not support it the 
negotiation phase fails within ensure_profile_and_level in 
gstvaapiencoder_h264.c where now the hardware is taken into account when 
calling ensure_hw_profile_limits. So for me it seems like the 
max_profile_idc is set based only on the vaapi encoder (implementation) 
limits, while it should also take into account the current hardware 
limits, shouldn't it? Or did I miss anything?
Maybe also the initial value within the negotiation phase shouldn't be 
GST_H264_PROFILE_STEREO_HIGH and the error is within the initial list of 
profiles. I'm not sure why get_available_caps(encode) seems to return 
GST_H264_PROFILE_STEREO_HIGH, while the later call to 
ensure_hw_profile_limits does not seem to support it.

I can workaround the problem for now by explicitly defining the profile 
used, but it seems like a bug to me.