On Fri Mar 24 09:25:44 2023 +0000, Rémi Bernon wrote:
As far as I know, it's all related to the media type `MF_MT_DEFAULT_STRIDE` attribute, and it being implicitly negative for RGB formats. And for legacy media type structures, such as `VIDEOFORMAT`, I believe there is a native translation to MF media type, using MF APIs, that we should mimic. YUV formats simply cannot have a negative stride, and it usually raises `MF_E_INVALID_MEDIA_TYPE` errors. Then, I'm wondering if we can have a better integration of signed strides in the code rather than relying on a ad-hoc `videoflip` element. The `wg_transform` has a buffer pool with metadata, and should be able to convey the plane alignment and stride requirements to the GStreamer elements. Whether this can translate a negative stride to a vertically flipped frame or not, I don't know, but I think it's worth exploring. This would improve compatibility with other possible stride values, as I believe we have some known issues already with some YUV formats using a different default stride on Windows and GStreamer. In any case I'd expect the stride sign to checked when deciding whether to flip or not the buffers, and I think it also needs to be taken into account in the plane padding computation. As far as the video processor tests goes, the padding is also usually reversed with RGB formats. And yes, it's definitely related to !2159.
some update to this draft: - you're right about the trigger based on stride sign; now using that - patch is now based on MR!2159 as it already does pushing the sign information to wg_transform
the color_video test still passes and the videoproc test (mf/tests/transform.c:6320) almost passes: - the good news: the % of error went down from 85% to 2%, - but I fear that something still needs care here: digging a bit further on the 2% distance: out of 96*96*3 bytes compared, 5834 have an error which is more or equal to 2 (and some greater than that), which looks suspicious from what should pure rounding discrepancy. Didn't look further.
however, I didn't find a simple way to trigger flip video in gstreamer. This doesn't mean there isn't. So, I kept using the explicit videoflip element in the pipeline.
What remains to be done: - simplify it from not storing as much information in struct wg_transform - theorically, we should pass the stride value if defined by caller (apart from the sign) to wg_transform.c. - analyze discrepancy on transform.c:videoproc test.
Further work on this patch is conditionned on how things settle down on 2159. So: ``` WaitForSingleEvent(MR!2159, INFINITE) ```