[PATCH v2 0/5] MR8144: winegstreamer: Add basic support for GST_VIDEO_FORMAT_I420_10LE.
Windows doesn't support I420 10LE, but uses P010 instead, thus GST_VIDEO_FORMAT_I420_10LE is ultimately converted to GST_VIDEO_FORMAT_P010_10LE. This fixes playback of videos in VRChat that use i420 10le as the raw video color format. -- v2: winedmo: Add support for the P010 format. winegstreamer: Add support for the I420 10LE format. winegstreamer: Add support for the P010 format. mfplat: Add support for the P010 format. mfplat/tests: Add image size tests for P010. https://gitlab.winehq.org/wine/wine/-/merge_requests/8144
From: Brendan McGrath <bmcgrath(a)codeweavers.com> --- dlls/mfplat/tests/mfplat.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/dlls/mfplat/tests/mfplat.c b/dlls/mfplat/tests/mfplat.c index 6fe3f40cd93..386b2d22144 100644 --- a/dlls/mfplat/tests/mfplat.c +++ b/dlls/mfplat/tests/mfplat.c @@ -5703,6 +5703,13 @@ image_size_tests[] = { &MFVideoFormat_IYUV, 4, 2, 12, 0, 384, 12, 128 }, { &MFVideoFormat_IYUV, 4, 3, 18, 0, 576, 18, 128 }, { &MFVideoFormat_IYUV, 320, 240, 115200, 0, 138240, 115200, 384 }, + + { &MFVideoFormat_P010, 1, 2, 12, 6, 192, 6, 64 }, + { &MFVideoFormat_P010, 2, 2, 12, 0, 192, 12, 64 }, + { &MFVideoFormat_P010, 2, 4, 24, 0, 384, 24, 64 }, + { &MFVideoFormat_P010, 3, 2, 24, 18, 192, 18, 64 }, + { &MFVideoFormat_P010, 4, 2, 24, 0, 192, 24, 64 }, + { &MFVideoFormat_P010, 320, 240, 230400, 0, 230400, 230400, 640 }, }; static void test_MFCalculateImageSize(void) @@ -5726,6 +5733,7 @@ static void test_MFCalculateImageSize(void) IsEqualGUID(ptr->subtype, &MFVideoFormat_ABGR32); hr = MFCalculateImageSize(ptr->subtype, ptr->width, ptr->height, &size); + todo_wine_if(i >= 140) ok(hr == S_OK || broken(is_broken && hr == E_INVALIDARG), "%u: failed to calculate image size, hr %#lx.\n", i, hr); if (hr == S_OK) { @@ -5761,6 +5769,7 @@ static void test_MFGetPlaneSize(void) hr = pMFGetPlaneSize(ptr->subtype->Data1, ptr->width, ptr->height, &size); ok(hr == S_OK, "%u: failed to get plane size, hr %#lx.\n", i, hr); + todo_wine_if(i >= 140) ok(size == plane_size, "%u: unexpected plane size %lu, expected %u. Size %u x %u, format %s.\n", i, size, plane_size, ptr->width, ptr->height, wine_dbgstr_an((char*)&ptr->subtype->Data1, 4)); } @@ -7529,8 +7538,16 @@ static void test_MFCreate2DMediaBuffer(void) winetest_push_context("%u, %u x %u, format %s", i, ptr->width, ptr->height, wine_dbgstr_guid(ptr->subtype)); hr = pMFCreate2DMediaBuffer(ptr->width, ptr->height, ptr->subtype->Data1, FALSE, &buffer); + todo_wine_if(i >= 140) ok(hr == S_OK, "Failed to create a buffer, hr %#lx.\n", hr); + if (hr != S_OK) + { + skip("P010 is not implemented in wine.\n"); + winetest_pop_context(); + continue; + } + hr = IMFMediaBuffer_GetMaxLength(buffer, &length); ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); ok(length == ptr->max_length, "Unexpected maximum length %lu.\n", length); @@ -7794,8 +7811,15 @@ static void test_MFCreate2DMediaBuffer(void) continue; hr = pMFCreate2DMediaBuffer(ptr->width, ptr->height, ptr->subtype->Data1, FALSE, &buffer); + todo_wine_if(i >= 140) ok(hr == S_OK, "Failed to create a buffer, hr %#lx.\n", hr); + if (hr != S_OK) + { + skip("P010 is not implemented in wine.\n"); + continue; + } + hr = IMFMediaBuffer_QueryInterface(buffer, &IID_IMF2DBuffer, (void **)&_2dbuffer); ok(hr == S_OK, "Failed to get interface, hr %#lx.\n", hr); -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/8144
From: Brendan McGrath <bmcgrath(a)codeweavers.com> --- dlls/mfplat/buffer.c | 5 +++++ dlls/mfplat/mediatype.c | 7 +++++++ dlls/mfplat/tests/mfplat.c | 17 ----------------- 3 files changed, 12 insertions(+), 17 deletions(-) diff --git a/dlls/mfplat/buffer.c b/dlls/mfplat/buffer.c index 65fab78d427..6d59245b40c 100644 --- a/dlls/mfplat/buffer.c +++ b/dlls/mfplat/buffer.c @@ -1405,6 +1405,7 @@ static p_copy_image_func get_2d_buffer_copy_func(DWORD fourcc) return copy_image_imc2; case MAKEFOURCC('N','V','1','2'): + case MAKEFOURCC('P','0','1','0'): return copy_image_nv12; default: @@ -1454,6 +1455,9 @@ static HRESULT create_2d_buffer(DWORD width, DWORD height, DWORD fourcc, BOOL bo case MAKEFOURCC('N','V','1','2'): plane_size = stride * height * 3 / 2; break; + case MAKEFOURCC('P','0','1','0'): + plane_size = width * height * 3; + break; default: plane_size = stride * height; } @@ -1492,6 +1496,7 @@ static HRESULT create_2d_buffer(DWORD width, DWORD height, DWORD fourcc, BOOL bo case MAKEFOURCC('N','V','1','1'): case MAKEFOURCC('I','4','2','0'): case MAKEFOURCC('I','Y','U','V'): + case MAKEFOURCC('P','0','1','0'): max_length = pitch * height * 3 / 2; break; default: diff --git a/dlls/mfplat/mediatype.c b/dlls/mfplat/mediatype.c index 755ef7e2216..10a92ad7464 100644 --- a/dlls/mfplat/mediatype.c +++ b/dlls/mfplat/mediatype.c @@ -2728,6 +2728,7 @@ static struct uncompressed_video_format video_formats[] = { &MFVideoFormat_IYUV, 12, 0, 0, 1, -1 }, { &MFVideoFormat_NV11, 12, 0, 0, 1, -1 }, { &MFVideoFormat_NV12, 12, 0, 0, 1, -1 }, + { &MFVideoFormat_P010, 24, 0, 0, 1, -1 }, { &MFVideoFormat_D16, 16, 3, 0, 0, -1 }, { &MFVideoFormat_L16, 16, 3, 0, 0, -1 }, { &MFVideoFormat_UYVY, 16, 0, 0, 1, -1 }, @@ -2762,6 +2763,7 @@ static struct uncompressed_video_format *mf_get_video_format(const GUID *subtype static unsigned int mf_get_stride_for_format(const struct uncompressed_video_format *format, unsigned int width) { if (format->bpp < 8) return (width * format->bpp) / 8; + else if (IsEqualGUID(format->subtype, &MFVideoFormat_P010)) return width * 2; return (width * (format->bpp / 8) + format->alignment) & ~format->alignment; } @@ -2831,6 +2833,10 @@ HRESULT WINAPI MFCalculateImageSize(REFGUID subtype, UINT32 width, UINT32 height /* 2 x 2 block, interleaving UV for half the height */ *size = ((width + 1) & ~1) * height * 3 / 2; break; + case MAKEFOURCC('P','0','1','0'): + /* width is rounded up to a multiple of two */ + *size = ((width + 1) & ~1) * height * 3; + break; case MAKEFOURCC('N','V','1','1'): *size = ((width + 3) & ~3) * height * 3 / 2; break; @@ -2875,6 +2881,7 @@ HRESULT WINAPI MFGetPlaneSize(DWORD fourcc, DWORD width, DWORD height, DWORD *si case MAKEFOURCC('I','4','2','0'): case MAKEFOURCC('I','Y','U','V'): case MAKEFOURCC('N','V','1','1'): + case MAKEFOURCC('P','0','1','0'): *size = stride * height * 3 / 2; break; default: diff --git a/dlls/mfplat/tests/mfplat.c b/dlls/mfplat/tests/mfplat.c index 386b2d22144..1cdbb6f48ff 100644 --- a/dlls/mfplat/tests/mfplat.c +++ b/dlls/mfplat/tests/mfplat.c @@ -5733,7 +5733,6 @@ static void test_MFCalculateImageSize(void) IsEqualGUID(ptr->subtype, &MFVideoFormat_ABGR32); hr = MFCalculateImageSize(ptr->subtype, ptr->width, ptr->height, &size); - todo_wine_if(i >= 140) ok(hr == S_OK || broken(is_broken && hr == E_INVALIDARG), "%u: failed to calculate image size, hr %#lx.\n", i, hr); if (hr == S_OK) { @@ -5769,7 +5768,6 @@ static void test_MFGetPlaneSize(void) hr = pMFGetPlaneSize(ptr->subtype->Data1, ptr->width, ptr->height, &size); ok(hr == S_OK, "%u: failed to get plane size, hr %#lx.\n", i, hr); - todo_wine_if(i >= 140) ok(size == plane_size, "%u: unexpected plane size %lu, expected %u. Size %u x %u, format %s.\n", i, size, plane_size, ptr->width, ptr->height, wine_dbgstr_an((char*)&ptr->subtype->Data1, 4)); } @@ -7538,16 +7536,8 @@ static void test_MFCreate2DMediaBuffer(void) winetest_push_context("%u, %u x %u, format %s", i, ptr->width, ptr->height, wine_dbgstr_guid(ptr->subtype)); hr = pMFCreate2DMediaBuffer(ptr->width, ptr->height, ptr->subtype->Data1, FALSE, &buffer); - todo_wine_if(i >= 140) ok(hr == S_OK, "Failed to create a buffer, hr %#lx.\n", hr); - if (hr != S_OK) - { - skip("P010 is not implemented in wine.\n"); - winetest_pop_context(); - continue; - } - hr = IMFMediaBuffer_GetMaxLength(buffer, &length); ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); ok(length == ptr->max_length, "Unexpected maximum length %lu.\n", length); @@ -7811,15 +7801,8 @@ static void test_MFCreate2DMediaBuffer(void) continue; hr = pMFCreate2DMediaBuffer(ptr->width, ptr->height, ptr->subtype->Data1, FALSE, &buffer); - todo_wine_if(i >= 140) ok(hr == S_OK, "Failed to create a buffer, hr %#lx.\n", hr); - if (hr != S_OK) - { - skip("P010 is not implemented in wine.\n"); - continue; - } - hr = IMFMediaBuffer_QueryInterface(buffer, &IID_IMF2DBuffer, (void **)&_2dbuffer); ok(hr == S_OK, "Failed to get interface, hr %#lx.\n", hr); -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/8144
From: Brendan McGrath <bmcgrath(a)codeweavers.com> --- dlls/winegstreamer/mfplat.c | 1 + dlls/winegstreamer/quartz_parser.c | 1 + dlls/winegstreamer/unixlib.h | 1 + dlls/winegstreamer/video_decoder.c | 1 + dlls/winegstreamer/wg_format.c | 3 +++ dlls/winegstreamer/wg_media_type.c | 1 + 6 files changed, 8 insertions(+) diff --git a/dlls/winegstreamer/mfplat.c b/dlls/winegstreamer/mfplat.c index fd2c6995d8c..4088a935625 100644 --- a/dlls/winegstreamer/mfplat.c +++ b/dlls/winegstreamer/mfplat.c @@ -183,6 +183,7 @@ video_formats[] = {&MFVideoFormat_I420, WG_VIDEO_FORMAT_I420}, {&MFVideoFormat_IYUV, WG_VIDEO_FORMAT_I420}, {&MFVideoFormat_NV12, WG_VIDEO_FORMAT_NV12}, + {&MFVideoFormat_P010, WG_VIDEO_FORMAT_P010_10LE}, {&MFVideoFormat_UYVY, WG_VIDEO_FORMAT_UYVY}, {&MFVideoFormat_YUY2, WG_VIDEO_FORMAT_YUY2}, {&MFVideoFormat_YV12, WG_VIDEO_FORMAT_YV12}, diff --git a/dlls/winegstreamer/quartz_parser.c b/dlls/winegstreamer/quartz_parser.c index 19b104d55df..67aaaa3e134 100644 --- a/dlls/winegstreamer/quartz_parser.c +++ b/dlls/winegstreamer/quartz_parser.c @@ -488,6 +488,7 @@ static const GUID *wg_video_format_get_mediasubtype(enum wg_video_format format) case WG_VIDEO_FORMAT_AYUV: return &MEDIASUBTYPE_AYUV; case WG_VIDEO_FORMAT_I420: return &MEDIASUBTYPE_I420; case WG_VIDEO_FORMAT_NV12: return &MEDIASUBTYPE_NV12; + case WG_VIDEO_FORMAT_P010_10LE: return &MEDIASUBTYPE_P010; case WG_VIDEO_FORMAT_UYVY: return &MEDIASUBTYPE_UYVY; case WG_VIDEO_FORMAT_YUY2: return &MEDIASUBTYPE_YUY2; case WG_VIDEO_FORMAT_YV12: return &MEDIASUBTYPE_YV12; diff --git a/dlls/winegstreamer/unixlib.h b/dlls/winegstreamer/unixlib.h index 83e38849fa9..179f15f78f7 100644 --- a/dlls/winegstreamer/unixlib.h +++ b/dlls/winegstreamer/unixlib.h @@ -104,6 +104,7 @@ enum wg_video_format WG_VIDEO_FORMAT_AYUV, WG_VIDEO_FORMAT_I420, WG_VIDEO_FORMAT_NV12, + WG_VIDEO_FORMAT_P010_10LE, WG_VIDEO_FORMAT_UYVY, WG_VIDEO_FORMAT_YUY2, WG_VIDEO_FORMAT_YV12, diff --git a/dlls/winegstreamer/video_decoder.c b/dlls/winegstreamer/video_decoder.c index a65f1781f96..bee035f1848 100644 --- a/dlls/winegstreamer/video_decoder.c +++ b/dlls/winegstreamer/video_decoder.c @@ -53,6 +53,7 @@ static const struct subtype_info subtype_info_list[] = { &MFVideoFormat_UYVY, 16, MAKEFOURCC('U', 'Y', 'V', 'Y') }, { &MFVideoFormat_YVYU, 16, MAKEFOURCC('Y', 'V', 'Y', 'U') }, { &MFVideoFormat_NV11, 12, MAKEFOURCC('N', 'V', '1', '1') }, + { &MFVideoFormat_P010, 24, MAKEFOURCC('P', '0', '1', '0') }, { &MFVideoFormat_RGB8, 8, BI_RGB }, { &MFVideoFormat_RGB555, 16, BI_RGB }, { &MFVideoFormat_RGB565, 16, BI_BITFIELDS }, diff --git a/dlls/winegstreamer/wg_format.c b/dlls/winegstreamer/wg_format.c index 12fee8c4923..027184e0846 100644 --- a/dlls/winegstreamer/wg_format.c +++ b/dlls/winegstreamer/wg_format.c @@ -153,6 +153,8 @@ static enum wg_video_format wg_video_format_from_gst(GstVideoFormat format) return WG_VIDEO_FORMAT_I420; case GST_VIDEO_FORMAT_NV12: return WG_VIDEO_FORMAT_NV12; + case GST_VIDEO_FORMAT_P010_10LE: + return WG_VIDEO_FORMAT_P010_10LE; case GST_VIDEO_FORMAT_UYVY: return WG_VIDEO_FORMAT_UYVY; case GST_VIDEO_FORMAT_YUY2: @@ -771,6 +773,7 @@ static GstVideoFormat wg_video_format_to_gst(enum wg_video_format format) case WG_VIDEO_FORMAT_YUY2: return GST_VIDEO_FORMAT_YUY2; case WG_VIDEO_FORMAT_YV12: return GST_VIDEO_FORMAT_YV12; case WG_VIDEO_FORMAT_YVYU: return GST_VIDEO_FORMAT_YVYU; + case WG_VIDEO_FORMAT_P010_10LE: return GST_VIDEO_FORMAT_P010_10LE; default: return GST_VIDEO_FORMAT_UNKNOWN; } } diff --git a/dlls/winegstreamer/wg_media_type.c b/dlls/winegstreamer/wg_media_type.c index 9e829f69680..44b39743bc3 100644 --- a/dlls/winegstreamer/wg_media_type.c +++ b/dlls/winegstreamer/wg_media_type.c @@ -653,6 +653,7 @@ static GUID subtype_from_gst_video_format(GstVideoFormat video_format) case GST_VIDEO_FORMAT_AYUV: return MFVideoFormat_AYUV; case GST_VIDEO_FORMAT_I420: return MFVideoFormat_I420; case GST_VIDEO_FORMAT_NV12: return MFVideoFormat_NV12; + case GST_VIDEO_FORMAT_P010_10LE: return MFVideoFormat_P010; case GST_VIDEO_FORMAT_UYVY: return MFVideoFormat_UYVY; case GST_VIDEO_FORMAT_YUY2: return MFVideoFormat_YUY2; case GST_VIDEO_FORMAT_YV12: return MFVideoFormat_YV12; -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/8144
From: Brendan McGrath <bmcgrath(a)codeweavers.com> Windows doesn't have support for the I420 10LE format. Instead, a decoding MFT will offer P010 as an output format. Therefore this change translates GST_VIDEO_FORMAT_I420_10LE to WG_VIDEO_FORMAT_P010_10LE so that Wine also offers P010 as an output. --- dlls/winegstreamer/wg_format.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/dlls/winegstreamer/wg_format.c b/dlls/winegstreamer/wg_format.c index 027184e0846..70255a39385 100644 --- a/dlls/winegstreamer/wg_format.c +++ b/dlls/winegstreamer/wg_format.c @@ -151,6 +151,8 @@ static enum wg_video_format wg_video_format_from_gst(GstVideoFormat format) return WG_VIDEO_FORMAT_AYUV; case GST_VIDEO_FORMAT_I420: return WG_VIDEO_FORMAT_I420; + case GST_VIDEO_FORMAT_I420_10LE: + return WG_VIDEO_FORMAT_P010_10LE; case GST_VIDEO_FORMAT_NV12: return WG_VIDEO_FORMAT_NV12; case GST_VIDEO_FORMAT_P010_10LE: -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/8144
From: Brendan McGrath <bmcgrath(a)codeweavers.com> --- dlls/winedmo/unix_media_type.c | 1 + 1 file changed, 1 insertion(+) diff --git a/dlls/winedmo/unix_media_type.c b/dlls/winedmo/unix_media_type.c index 28cc27edb03..fac3a82e8c9 100644 --- a/dlls/winedmo/unix_media_type.c +++ b/dlls/winedmo/unix_media_type.c @@ -212,6 +212,7 @@ static GUID subtype_from_pixel_format( enum AVPixelFormat fmt ) case AV_PIX_FMT_BGR24: return MFVideoFormat_RGB24; case AV_PIX_FMT_NV12: return MFVideoFormat_NV12; case AV_PIX_FMT_NV21: return MFVideoFormat_NV21; + case AV_PIX_FMT_P010: return MFVideoFormat_P010; case AV_PIX_FMT_RGB565: return MFVideoFormat_RGB565; case AV_PIX_FMT_RGB555: return MFVideoFormat_RGB555; case AV_PIX_FMT_RGB8: return MFVideoFormat_RGB8; -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/8144
On Tue May 27 09:26:07 2025 +0000, Rémi Bernon wrote:
Please add the format to the list of tested formats in `dlls/mfplat/tests/mfplat.c`, and then probably to the `uncompressed_video_format` array in `dlls/mfplat/mediatype.c`. Also this would probably also need to be added to the `subtype_from_gst_video_format` mapping in `dlls/winegstreamer/wg_media_type.c` and `subtype_from_pixel_format` in `dlls/winedmo/unix_media_type.c`. Also perhaps would be good to include it in the `subtype_info_list` table in `dlls/winegstreamer/video_decoder.c`, although it's not used by any of our decoders yet. Done... I think. I didn't include a decoding test, as I couldn't find an encoder that actually supported the I420 10LE format. Although, I only looked through GStreamer. It's possible FFMpeg offers one. Let me know if you want a test for that.
-- https://gitlab.winehq.org/wine/wine/-/merge_requests/8144#note_104645
Can we please keep the formats in alphabetical order?
Done
I'm confused, though, in what situation do we even need to handle I420_10LE at all, if we're going to force P010_10LE anyway?
The decoder we use (`av1dec`) doesn't offer `P010_10LE` as an output. We need to use `videoconvert` for that. But Windows doesn't support `I420_10LE`, instead it offers `P010_10LE`. So we need to set the caps on `videoconvert` to `P010_10LE`. We do that at the moment through `wg_parser_stream_enable` which sets the `desired_caps` and creates a reconfigure event.
Translating I420_10LE to P010_10LE in wg_video_format_from_gst() doesn't seem like a very clear way to do things.
What we are doing is offering `P010_10LE` as an output when GStreamer states the video format is `I420_10LE`. So if the application selects `P010_10LE` as the desired output, we'll use `P010_10LE` as the `desired_caps`. I've separated this part out in to its own commit and I think I have now better explained it in the commit message. -- https://gitlab.winehq.org/wine/wine/-/merge_requests/8144#note_104653
On Tue May 27 10:12:44 2025 +0000, Brendan McGrath wrote:
Can we please keep the formats in alphabetical order? Done I'm confused, though, in what situation do we even need to handle I420_10LE at all, if we're going to force P010_10LE anyway? The decoder we use (`av1dec`) doesn't offer `P010_10LE` as an output. We need to use `videoconvert` for that. But Windows doesn't support `I420_10LE`, instead it offers `P010_10LE`. So we need to set the caps on `videoconvert` to `P010_10LE`. We do that at the moment through `wg_parser_stream_enable` which sets the `desired_caps` and creates a reconfigure event. Translating I420_10LE to P010_10LE in wg_video_format_from_gst() doesn't seem like a very clear way to do things. What we are doing is offering `P010_10LE` as an output when GStreamer states the video format is `I420_10LE`. So if the application selects `P010_10LE` as the desired output, we'll use `P010_10LE` as the `desired_caps`. I've separated this part out in to its own commit and I think I have now better explained it in the commit message. GStreamer doesn't "state" the video format is anything, though. We treat the default format it outputs as "preferred" format, but there isn't actually a sense in which I420_10LE is the correct format. In fact the mfplat frontend ignores the preferred format and substitutes its own list, largely because mfplat applications are especially picky about which formats get enumerated.
The confusion probably comes from 08730dae2, which was committed over my objections. -- https://gitlab.winehq.org/wine/wine/-/merge_requests/8144#note_104709
On Tue May 27 20:20:41 2025 +0000, Elizabeth Figura wrote:
GStreamer doesn't "state" the video format is anything, though. We treat the default format it outputs as "preferred" format, but there isn't actually a sense in which I420_10LE is the correct format. In fact the mfplat frontend ignores the preferred format and substitutes its own list, largely because mfplat applications are especially picky about which formats get enumerated. The confusion probably comes from 08730dae2, which was committed over my objections. Sorry, you're right. When I say "state" I guess I'm referring to what comes out of the decoder (which is always `I420_10LE`). But yes, then the translation of `I420_10LE` to `P010_10LE` that is passed to the application is impacting only the preferred format. But I think we're on the same page.
-- https://gitlab.winehq.org/wine/wine/-/merge_requests/8144#note_104713
On Tue May 27 20:56:55 2025 +0000, Brendan McGrath wrote:
Sorry, you're right. When I say "state" I guess I'm referring to what comes out of the decoder (which is always `I420_10LE`). But yes, then the translation of `I420_10LE` to `P010_10LE` that is passed to the application is impacting only the preferred format. But I think we're on the same page. At the risk of too much information, I'll share a few more details of what I'm looking at.
The file I'm testing against is: https://dragon.theater/tmp/av1test.mp4 If I run `ffprobe` against this, I get: ``` Video: av1 (Main) (av01 / 0x31307661), yuv420p10le(tv, progressive) ``` If I check the output of `IMFTransform::GetOutputAvailableType` on the Windows AV1 decoder MFT, I get: ``` Output 0: - {F7E34C9A-42E8-4714-B74B-CB29D72C35E5} (MF_MT_SUBTYPE): {30313050-0000-0010-8000-00AA00389B71} (MFVideoFormat_P010) Output 1: - {F7E34C9A-42E8-4714-B74B-CB29D72C35E5} (MF_MT_SUBTYPE): {36313050-0000-0010-8000-00AA00389B71} (MFVideoFormat_P016) Output 2: - {F7E34C9A-42E8-4714-B74B-CB29D72C35E5} (MF_MT_SUBTYPE): {3231564E-0000-0010-8000-00AA00389B71} (MFVideoFormat_NV12) Output 3: - {F7E34C9A-42E8-4714-B74B-CB29D72C35E5} (MF_MT_SUBTYPE): {56555949-0000-0010-8000-00AA00389B71} (MFVideoFormat_IYUV) ``` So you can see `ffprobe` states `yuv420p10le`, but that's not offered by Windows. Instead `MFVideoFormat_P010` is the preferred option. Under Wine (when using a winegstreamer source), we don't use an MFT to decode, that is done by the media source. With this MR, the offered outputs of the media source are: ``` Output 0: - {F7E34C9A-42E8-4714-B74B-CB29D72C35E5} (MF_MT_SUBTYPE): {30313050-0000-0010-8000-00AA00389B71} (MFVideoFormat_P010) Output 1: - {F7E34C9A-42E8-4714-B74B-CB29D72C35E5} (MF_MT_SUBTYPE): {3231564E-0000-0010-8000-00AA00389B71} (MFVideoFormat_NV12) Output 2: - {F7E34C9A-42E8-4714-B74B-CB29D72C35E5} (MF_MT_SUBTYPE): {32315659-0000-0010-8000-00AA00389B71} (MFVideoFormat_YV12) Output 3: - {F7E34C9A-42E8-4714-B74B-CB29D72C35E5} (MF_MT_SUBTYPE): {32595559-0000-0010-8000-00AA00389B71} (MFVideoFormat_YUY2) Output 4: - {F7E34C9A-42E8-4714-B74B-CB29D72C35E5} (MF_MT_SUBTYPE): {30323449-0000-0010-8000-00AA00389B71} (MFVideoFormat_I420) Output 5: - {F7E34C9A-42E8-4714-B74B-CB29D72C35E5} (MF_MT_SUBTYPE): {56555949-0000-0010-8000-00AA00389B71} (MFVideoFormat_IYUV) ``` So, as discussed, `MFVideoFormat_P010` is just promoted to being the preferred option (Option 0). And finally, attached is the GStreamer pipeline when I play this file with MF Media Engine under Wine (with this MR in place). [0.00.00.159525593-wg_parser_caps.dot](/uploads/cb2c798b24b2f24819dee4c02d92fa40/0.00.00.159525593-wg_parser_caps.dot) You can see `av1dec` outputs `I420_10LE`. And we don't (can't) change that (which is what I meant by "state"). It's the very last element (`videoconvert1`) that converts it to `NV12`. -- https://gitlab.winehq.org/wine/wine/-/merge_requests/8144#note_104741
On Tue May 27 21:27:36 2025 +0000, Brendan McGrath wrote:
At the risk of too much information, I'll share a few more details of what I'm looking at. The file I'm testing against is: https://dragon.theater/tmp/av1test.mp4 If I run `ffprobe` against this, I get: ``` Video: av1 (Main) (av01 / 0x31307661), yuv420p10le(tv, progressive) ``` If I check the output of `IMFTransform::GetOutputAvailableType` on the Windows AV1 decoder MFT, I get: ``` Output 0: - {F7E34C9A-42E8-4714-B74B-CB29D72C35E5} (MF_MT_SUBTYPE): {30313050-0000-0010-8000-00AA00389B71} (MFVideoFormat_P010) Output 1: - {F7E34C9A-42E8-4714-B74B-CB29D72C35E5} (MF_MT_SUBTYPE): {36313050-0000-0010-8000-00AA00389B71} (MFVideoFormat_P016) Output 2: - {F7E34C9A-42E8-4714-B74B-CB29D72C35E5} (MF_MT_SUBTYPE): {3231564E-0000-0010-8000-00AA00389B71} (MFVideoFormat_NV12) Output 3: - {F7E34C9A-42E8-4714-B74B-CB29D72C35E5} (MF_MT_SUBTYPE): {56555949-0000-0010-8000-00AA00389B71} (MFVideoFormat_IYUV) ``` So you can see `ffprobe` states `yuv420p10le`, but that's not offered by Windows. Instead `MFVideoFormat_P010` is the preferred option. Under Wine (when using a winegstreamer source), we don't use an MFT to decode, that is done by the media source. With this MR, the offered outputs of the media source are: ``` Output 0: - {F7E34C9A-42E8-4714-B74B-CB29D72C35E5} (MF_MT_SUBTYPE): {30313050-0000-0010-8000-00AA00389B71} (MFVideoFormat_P010) Output 1: - {F7E34C9A-42E8-4714-B74B-CB29D72C35E5} (MF_MT_SUBTYPE): {3231564E-0000-0010-8000-00AA00389B71} (MFVideoFormat_NV12) Output 2: - {F7E34C9A-42E8-4714-B74B-CB29D72C35E5} (MF_MT_SUBTYPE): {32315659-0000-0010-8000-00AA00389B71} (MFVideoFormat_YV12) Output 3: - {F7E34C9A-42E8-4714-B74B-CB29D72C35E5} (MF_MT_SUBTYPE): {32595559-0000-0010-8000-00AA00389B71} (MFVideoFormat_YUY2) Output 4: - {F7E34C9A-42E8-4714-B74B-CB29D72C35E5} (MF_MT_SUBTYPE): {30323449-0000-0010-8000-00AA00389B71} (MFVideoFormat_I420) Output 5: - {F7E34C9A-42E8-4714-B74B-CB29D72C35E5} (MF_MT_SUBTYPE): {56555949-0000-0010-8000-00AA00389B71} (MFVideoFormat_IYUV) ``` So, as discussed, `MFVideoFormat_P010` is just promoted to being the preferred option (Option 0). And finally, attached is the GStreamer pipeline when I play this file with MF Media Engine under Wine (with this MR in place). [0.00.00.159525593-wg_parser_caps.dot](/uploads/cb2c798b24b2f24819dee4c02d92fa40/0.00.00.159525593-wg_parser_caps.dot) You can see `av1dec` outputs `I420_10LE`. And we don't (can't) change that (which is what I meant by "state"). It's the very last element (`videoconvert1`) that converts it to `NV12`. Oh I'm wrong, mfplat does use the preferred format, I was also led astray by the same commit.
I think the best way to deal with that would either be to skip adding the preferred format to the list if it's UNKNOWN, or to add a WG_VIDEO_FORMAT_I420_10LE which is translated to P010 in wg_parser_get_preferred_format(). But converting I420_10LE to P010 in caps conversion doesn't seem like a good solution. -- https://gitlab.winehq.org/wine/wine/-/merge_requests/8144#note_104834
I think the best way to deal with that would either be to skip adding the preferred format to the list if it's UNKNOWN, or to add a WG_VIDEO_FORMAT_I420_10LE which is translated to P010 in wg_parser_get_preferred_format
I would prefer to keep adding the preferred option (i.e. translate to P010) as this more closely matches Windows behavior (and who knows, some day we may find an application dependent on it). But I would like to seek a bit more clarity about what we want (from both yourself and @rbernon). It sounds like we want to distinguish between the preferred and current formats. So I guess on the unix side, we want to record the format from the decoder and then we would translate that to a supported Windows format GUID and supply that as the _preferred_ format on the DLL side (which up until now has been one-to-one between `GST_VIDEO_FORMAT` and GUID). I also want to understand the role of the `WG_VIDEO_FORMAT` defines. Initially I thought it might be so we didn't need to include the GUIDs on the unix side, but I see we map directly from `GST_VIDEO_FORMAT` to GUID in `subtype_from_gst_video_format`. So I figured maybe it might be for mapping `GST_VIDEO_FORMAT` to a supported `WG_VIDEO_FORMAT`, so that `WG_VIDEO_FORMAT` could then be mapped one-to-one with a Windows GUID. It was this thought that prompted me to map it in `wg_video_format_from_gst`. But it sounds like we want to maintain a one-to-one relationship between `GST_VIDEO_FORMAT` and `WG_VIDEO_FORMAT`; so now I'm not sure of the `WG_VIDEO_FORMAT` defines purpose. -- https://gitlab.winehq.org/wine/wine/-/merge_requests/8144#note_104894
Under Wine (when using a winegstreamer source), we don't use an MFT to decode, that is done by the media source.
This is pretty much the problem, and it needs to be changed. This is what winedmo does already and I have no objection to make the winegstreamer media source do the same. The MF pipelines should be working well now as they are used for most of the MF media playback, assuming we pull any remaining Proton patches in the MF modules (which I don't think are many left). -- https://gitlab.winehq.org/wine/wine/-/merge_requests/8144#note_104949
I think the best way to deal with that would either be to skip adding the preferred format to the list if it's UNKNOWN, or to add a WG_VIDEO_FORMAT_I420_10LE which is translated to P010 in wg_parser_get_preferred_format
I would prefer to keep adding the preferred option (i.e. translate to P010) as this more closely matches Windows behavior (and who knows, some day we may find an application dependent on it).
I don't see how? I'm talking about internals here. Translating I420_10LE to P010 in wg_parser_get_preferred_format() would achieve the same thing on the win32 side, just in a more declarative way.
It sounds like we want to distinguish between the preferred and current formats. So I guess on the unix side, we want to record the format from the decoder and then we would translate that to a supported Windows format GUID and supply that as the _preferred_ format on the DLL side (which up until now has been one-to-one between `GST_VIDEO_FORMAT` and GUID).
Sure. To be clear, "preferred format" is exclusively a winegstreamer concept, not a win32 concept nor a GStreamer concept. It's just the format that GStreamer outputs as a first choice, so we assume it's a format that will require no extra conversion between raw formats, so it's the format we prefer to output if we have the choice. That just means it's the format we enumerate first in win32. In DirectShow, at least, this usually works since applications are less picky about the formats we offer by default. In Media Foundation the same is probably true in general, but unfortunately applications are more inclined to make assumptions and so it's not really a viable strategy.
I also want to understand the role of the `WG_VIDEO_FORMAT` defines. Initially I thought it might be so we didn't need to include the GUIDs on the unix side, but I see we map directly from `GST_VIDEO_FORMAT` to GUID in `subtype_from_gst_video_format`. So I figured maybe it might be for mapping `GST_VIDEO_FORMAT` to a supported `WG_VIDEO_FORMAT`, so that `WG_VIDEO_FORMAT` could then be mapped one-to-one with a Windows GUID. It was this thought that prompted me to map it in `wg_video_format_from_gst`. But it sounds like we want to maintain a one-to-one relationship between `GST_VIDEO_FORMAT` and `WG_VIDEO_FORMAT`; so now I'm not sure of the `WG_VIDEO_FORMAT` defines purpose.
There are a couple reasons that I didn't use GUIDs and still wouldn't change the code to use them: * wg_* was meant to be a neutral API between the frontends (then quartz and mfplat, and now also wmvcore). mfplat and quartz do not actually use the same GUIDs for video or audio types. * Moreover, I don't believe that either mfplat or quartz's set actually has all of the formats needed by the other. * A simple enumeration is just easier to deal with anyway, e.g. it can be switched over.
Under Wine (when using a winegstreamer source), we don't use an MFT to decode, that is done by the media source.
This is pretty much the problem, and it needs to be changed. This is what winedmo does already and I have no objection to make the winegstreamer media source do the same.
Well, I tried... -- https://gitlab.winehq.org/wine/wine/-/merge_requests/8144#note_105088
Translating I420_10LE to P010 in wg_parser_get_preferred_format() would achieve the same thing on the win32 side, just in a more declarative way.
I'm happy with that. I meant I'd prefer to offer P010 rather than skip adding the preferred format.
To be clear, "preferred format" is exclusively a winegstreamer concept, not a win32 concept nor a GStreamer concept.
I guess the parallel I'm drawing is from the following line in the documentation for [IMFTransform::GetOutputAvailableType](https://learn.microsoft.com/en-us/windows/win32/api/mftransform/nf-mftransfo...): _The MFT defines a list of available media types for each output stream and orders them by preference._ P010 is offered as the first option on Windows for the video I'm testing; and that's the behavior I wanted to keep. Although, my motive for that was in case an application was dependent on that behavior, but it looks like that list is only available through the MFT interface anyway (i.e. it's not mirrored by the IMFSourceReader interface for example, which only offers the native types), so maybe it's not so important.
There are a couple reasons that I didn't use GUIDs ...
OK, that makes sense. Thank you.
This is pretty much the problem, and it needs to be changed.
I agree, we want to match Windows behavior where we can, but I don't think that change is within the scope of this MR. So given my only motive for translating I420_10LE to P010 seems moot (as that behavior on Windows is in the AV1 decoder MFT, which we don't have), maybe just skipping the preferred format if it's UNKNOWN is the best option? -- https://gitlab.winehq.org/wine/wine/-/merge_requests/8144#note_105093
I agree, we want to match Windows behavior where we can, but I don't think that change is within the scope of this MR.
Well the scope of the MR could be to add support for the P010 format, which will be useful to implement that AV1 decoder. IMO you could drop the one I420_10LE to P010 mapping hunk for now (or replace it with a mapping to GST_VIDEO_FORMAT_P010_10LE). -- https://gitlab.winehq.org/wine/wine/-/merge_requests/8144#note_105099
participants (4)
-
Brendan McGrath -
Brendan McGrath (＠redmcg)
-
Elizabeth Figura (＠zfigura)
-
Rémi Bernon