[PATCH v5 0/2] MR10321: mfsrcsnk: Parse pixel aspect ratio from the avcC atom.
More specifically the SPS bytes in the avcC atom. Note this only fixes the problem for winedmo but not gstreamer. This is because gstreamer 1) does not parse the SPS, 2) will set pixel-aspect-ratio in caps regardless if a pasp atom is present. So there is no way for us to distinguish between 1) pasp being present and having an aspect ratio of 1:1, and 2) pasp not being present but pixel-aspect-ratio having been set to 1:1 by gstreamer as fallback. And because this is the case, I made sure winedmo is tested separately. In winedmo's case, if pasp is not present, we can set the PixelAspectRatio.Denominator in MFVIDEOFORMAT to 0 to distinguish. -- v5: mfsrcsnk: Parse pixel aspect ratio from the avcC atom. https://gitlab.winehq.org/wine/wine/-/merge_requests/10321
From: Yuxuan Shui <yshui@codeweavers.com> --- dlls/mfsrcsnk/tests/mfsrcsnk.c | 83 +++++++++++++++++++++++++ dlls/mfsrcsnk/tests/resource.rc | 15 +++++ dlls/mfsrcsnk/tests/test_metadata.mp4 | Bin 0 -> 1832 bytes dlls/mfsrcsnk/tests/test_metadata2.mp4 | Bin 0 -> 1848 bytes 4 files changed, 98 insertions(+) create mode 100644 dlls/mfsrcsnk/tests/test_metadata.mp4 create mode 100644 dlls/mfsrcsnk/tests/test_metadata2.mp4 diff --git a/dlls/mfsrcsnk/tests/mfsrcsnk.c b/dlls/mfsrcsnk/tests/mfsrcsnk.c index 527c0051050..d11c681e639 100644 --- a/dlls/mfsrcsnk/tests/mfsrcsnk.c +++ b/dlls/mfsrcsnk/tests/mfsrcsnk.c @@ -824,6 +824,88 @@ static void test_thinning(void) IMFMediaSource_Release(source); } +static void test_mp4_extra_metadata(void) +{ + IMFPresentationDescriptor *presentation_descriptor; + IMFStreamDescriptor *stream_descriptor; + IMFMediaTypeHandler *type_handler; + IMFMediaType *media_type; + IMFMediaSource *source; + IMFByteStream *stream; + DWORD stream_count; + UINT64 value64; + BOOL selected; + HRESULT hr; + + /* Some metadata are duplicated at multiple levels in a mp4. Notably, the pixel aspect ratio + * is defined in both: 1) the pasp atom, and 2) the avcC atom. + * This test checks how they are treated by the byte stream handler. For example: are they both + * parsed? Which one takes precedence if both are present? */ + + /* 1. has no pasp atom. Aspect ratio set to 3:4 in the SPS contained by the avcC atom. */ + stream = create_resource_byte_stream(L"test_metadata.mp4"); + hr = create_source(&CLSID_MPEG4ByteStreamHandlerPlugin, stream, &source); + IMFByteStream_Release(stream); + + if (FAILED(hr)) + { + win_skip("Failed to create MPEG4 source: %#lx.\n", hr); + return; + } + + hr = IMFMediaSource_CreatePresentationDescriptor(source, &presentation_descriptor); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + hr = IMFPresentationDescriptor_GetStreamDescriptorCount(presentation_descriptor, &stream_count); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + ok(stream_count == 1, "Unexpected stream count %lu.\n", stream_count); + hr = IMFPresentationDescriptor_GetStreamDescriptorByIndex(presentation_descriptor, 0, &selected, &stream_descriptor); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + hr = IMFStreamDescriptor_GetMediaTypeHandler(stream_descriptor, &type_handler); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + hr = IMFMediaTypeHandler_GetCurrentMediaType(type_handler, &media_type); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + value64 = 0xdeadbeef; + hr = IMFMediaType_GetUINT64(media_type, &MF_MT_PIXEL_ASPECT_RATIO, &value64); + todo_wine ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + todo_wine ok(value64 == 0x300000004, "Unexpected aspect ratio %#I64x.\n", value64); + IMFMediaType_Release(media_type); + IMFMediaTypeHandler_Release(type_handler); + IMFStreamDescriptor_Release(stream_descriptor); + IMFPresentationDescriptor_Release(presentation_descriptor); + hr = IMFMediaSource_Shutdown(source); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + IMFMediaSource_Release(source); + + /* 2. has pasp atom. Aspect ratio set to 3:4 in the SPS, and 15:16 in the pasp. */ + stream = create_resource_byte_stream(L"test_metadata2.mp4"); + hr = create_source(&CLSID_MPEG4ByteStreamHandlerPlugin, stream, &source); + IMFByteStream_Release(stream); + + hr = IMFMediaSource_CreatePresentationDescriptor(source, &presentation_descriptor); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + hr = IMFPresentationDescriptor_GetStreamDescriptorCount(presentation_descriptor, &stream_count); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + ok(stream_count == 1, "Unexpected stream count %lu.\n", stream_count); + hr = IMFPresentationDescriptor_GetStreamDescriptorByIndex(presentation_descriptor, 0, &selected, &stream_descriptor); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + hr = IMFStreamDescriptor_GetMediaTypeHandler(stream_descriptor, &type_handler); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + hr = IMFMediaTypeHandler_GetCurrentMediaType(type_handler, &media_type); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + value64 = 0xdeadbeef; + hr = IMFMediaType_GetUINT64(media_type, &MF_MT_PIXEL_ASPECT_RATIO, &value64); + todo_wine ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + /* pasp takes precedence */ + todo_wine ok(value64 == 0xf00000010, "Unexpected aspect ratio: %#I64x.\n", value64); + IMFMediaType_Release(media_type); + IMFMediaTypeHandler_Release(type_handler); + IMFStreamDescriptor_Release(stream_descriptor); + IMFPresentationDescriptor_Release(presentation_descriptor); + hr = IMFMediaSource_Shutdown(source); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + IMFMediaSource_Release(source); +} + static void test_end_of_presentation(void) { IMFPresentationDescriptor *pd; @@ -929,6 +1011,7 @@ START_TEST(mfsrcsnk) test_wave_sink(); test_thinning(); + test_mp4_extra_metadata(); test_end_of_presentation(); hr = MFShutdown(); diff --git a/dlls/mfsrcsnk/tests/resource.rc b/dlls/mfsrcsnk/tests/resource.rc index 7b3a6532f0a..89d92ea3241 100644 --- a/dlls/mfsrcsnk/tests/resource.rc +++ b/dlls/mfsrcsnk/tests/resource.rc @@ -29,3 +29,18 @@ */ /* @makedep: test_thinning.avi */ test_thinning.avi RCDATA test_thinning.avi + +/* Generated with: + ffmpeg -filter_complex "smptebars=s=64x64[vout]" -map "[vout]" -t 0.25 -r 25 -vcodec h264 \ + -brand mp42 -aspect 3:4 -an test.mp4 + + Then, manually remove the pasp atom from the generated mp4 file. + */ +/* @makedep: test_metadata.mp4 */ +test_metadata.mp4 RCDATA test_metadata.mp4 + +/* Same as above, but instead of removing the pasp, it's modify to have an aspect ratio of 15:16 + * (it was 3:4 originally) + */ +/* @makedep: test_metadata2.mp4 */ +test_metadata2.mp4 RCDATA test_metadata2.mp4 diff --git a/dlls/mfsrcsnk/tests/test_metadata.mp4 b/dlls/mfsrcsnk/tests/test_metadata.mp4 new file mode 100644 index 0000000000000000000000000000000000000000..9619569ed4b554a0c8a98bfef25a02c0fb1d73d1 GIT binary patch literal 1832 zcmZQzU{FXasVvAXFfn3aU}6B#nZ@}=iDk)#xdkSM3=AxmxhaVy3=9k$<(Vm|3=B*Q z*jWGnpL56c<=$sEJ+@ENxuI}V`Q7^pBQp~PU4`WQqErP#Fw;ZN$jn6FH^9|h*F?cF z%vm8hKP5F;L07>!zo0TFHLXO!$iTo@*T}%gR6$oEqokz3N?*Ucyj-s=GbJ@YCoxYi zzbIWFWUgLDNp6mUu0no6NoIatv6Vt{Vp3wVt)W6uYMQOFLP}~<PJVK>t)Z2Hm4QNH zUSdvVajLCBg|U@Eg`uIbLT;*UMrwsZacNR+s;#*~L2;$6A&87GO0hN6Gcd3+&@(Vl z$jz)sO^FAYXsD2z8ef!{m!4{CXr_>yQIwyX7@wPJYp76Cl$w*1S!`>lU{PU_l3ZeI zsE}NkYipp8lA4%Om7kYtYh<WnXsD2uSX>fcP@J7v08#-lps*l5KP{~|wZvA}NTDR7 zC^a#q*w#oPCqF+sF(WlGB_1Ybs8F1fnVgCcFi^-V0^5<CnVVPwaz|!fNor9}VsdJV zt${*PPH9nMWqfjeZb4#+t${*veqM1&QDSCZYD#=&UP)0RNKIN%Vs2`&t+7H<d_iSV zVs2)Nt&u`fd}2ys0mv6g@kyD9#UM*Ei&DY<Do@Q!&nQW<HB`tiNX?5+&o8hwP=E*) z*cvHhr&ea>mDn1Y8bF!xxtV#kMy3kI$*Fm%$)zQ>CI$*%+d)BElv<o&YoJh+91r&{ zNWLi9Hn}KGAvdX{C>7+b<f1fNBV#=Sg+h>LYz_3x3=|3rKn8>9#0pz8ONGLM;*!(? zTN8!Mg7~7ulFWQtLp>7%g~UQzLn}~hGBhwSFxsbfv@nEUUZQQ={?9;bW<x;Te~Dbz z<O$5}=k`5#Tpb~Hbi?<W`e{u|4~V{)|DvmbgW<_BN8f*UxN?L8UYZ`QG4_$owO*-s zo$>S<Z=t6<^it%@7r8GEv`IRrd(64?fAXp%r{}>=#h3kh#r_5!vrov8I}<#+z<$<& zNsDqPO%C1@sbp~`egnHd!_B1{zm#qr?c8-%aR%>g@fG4vpDfP0m;UVeCjKkT?zf!} z`Fin}zm^GpB`0?{!RWv4{q@4Tr>n{twXeH)$UglpPv%5PX-2Wa>Hhhf1E+dkog<~a zB7c5&a1opNP1Ox&>|Ec^zdhk>Qp1Z4%vRlXH3!o+hVK7zL;U^0O-oa>7M@F4mM;?| zelyG3A-efmmUqs}u>Z{4-rf=M{T=vb)q+~H^B1oz_{$oqeyHQh*6DwQvlC0MoN@f^ zR^JH9?vArma@^}VIKb?AP8EIsgB(F@#(9Y)PVx~B5O#8wlY9h7oX>Gqs^<#FjF~<h z&i^@gGB7Zm&&|&-V_;y&$t}xBVL$=QFBljYJ}@vcfDl545y1qr(D)!L!Q$8nMg|6^ z=#rwuYz78~n38O;oyaD@*dLI!g9X6^k|-J<Y%hoaVFt$Uxha{6pn@|u1#AbBMk59W z1|EjcBVZxLjFg-rFpGhK0aU2L*if2rU2bMx8Uq7^NLenrK@2J>5ILEYqEwI?kRFh; zcuI?M6d1ta#+X-Jl9a>1z%Z}4q!<>!AT@oU(i@}!q7KS}Q4S0a3?2*&U<?ui=>XBV zvBdxX3=9kwiDk*oj42FU{~t2Qr>8J*t-0x<_WuCG0Y<4tMy3S}49pA)3=C|Jk6bn# zU}Rv+c>Vaa(g6kr28rU5l47XgAiLN=^$Y_8gF`YxoDJeq7LYPV1_mxD4da8jpp?P{ zVlXl=fW<|MONx=r1(_{VTvD72QUoy%q=x}&HiJ%aNpTfa4n(mrFfcy{#}_Eh5LgHl OBgG}j`5;A5a|{5QSmr<g literal 0 HcmV?d00001 diff --git a/dlls/mfsrcsnk/tests/test_metadata2.mp4 b/dlls/mfsrcsnk/tests/test_metadata2.mp4 new file mode 100644 index 0000000000000000000000000000000000000000..be4f6a7eb1d47721751982331bd32c22de128465 GIT binary patch literal 1848 zcmZQzU{FXasVvAXFfn3aU}6B#nZ@}=iDk)#xdkSM3=AxmxhaVy3=9k$<(Vm|3=B*Q z*jWGnpL56c<=$sEJ+@ENxuI}V`Q7^pBQp~PU4`WQqErP#Fw;ZN$jn6FH^9|h*F?cF z%vm8hKP5F;L07>!zo0TFHLXO!$iTo@*T}%gR6$oEqokz3N?*Ucyj-s=GbJ@YCoxYi zzbIWFWUgLDNp6mUu0no6NoIatv6Vt{Vp3wVt)W6uYMQOFLP}~<PJVK>t)Z2Hm4QNH zUSdvVajLCBg|U@Eg`uIbLT;*UMrwsZacNR+s;#*~L2;$6A&87GO0hN6Gcd3+&@(Vl z$jz)sO^FAYXsD2z8ef!{m!4{CXr_>yQIwyX7@wPJYp76Cl$w*1S!`>lU{PU_l3ZeI zsE}NkYipp8lA4%Om7kYtYh<WnXsD2uSX>fcP@J7v08#-lps*l5KP{~|wZvA}NTDR7 zC^a#q*w#oPCqF+sF(WlGB_1Ybs8F1fnVgCcFi^-V0^5<CnVVPwaz|!fNor9}VsdJV zt${*PPH9nMWqfjeZb4#+t${*veqM1&QDSCZYD#=&UP)0RNKIN%Vs2`&t+7H<d_iSV zVs2)Nt&u`fd}2ys0mv6g@kyD9#UM*Ei&DY<Do@Q!&nQW<HB`tiNX?5+&o8hwP=E*) z*cvHhr&ea>mDn1Y8bF!xxtV#kMy3kI$*Fm%$)zQ>CI$*%+d)BElv<o&YoJh+91r&{ zNWLi9Hn}KGAvdX{C>7+b<f1fNBV#=Sg+h>LYz_3x3=|3rKn8>9#0pz8ONGLM;*!(? zTN8!Mg7~7ulFWQtLp>7%g~UQzLn}~hGBhwSFxsbfv@nEUUZQQ={?9;bW<x;Te~Dbz z<O$5}=k`5#Tpb~Hbi?<W`e{u|4~V{)|DvmbgW<_BN8f*UxN?L8UYZ`QG4_$owO*-s zo$>S<Z=t6<^it%@7r8GEv`IRrd(64?fAXp%r{}>=#h3kh#r_5!vrov8I}<#+z<$<& zNsDqPO%C1@sbp~`egnHd!_B1{zm#qr?c8-%aR%>g@fG4vpDfP0m;UVeCjKkT?zf!} z`Fin}zm^GpB`0?{!RWv4{q@4Tr>n{twXeH)$UglpPv%5PX-2Wa>Hhhf1E+dkog<~a zB7c5&a1opNP1Ox&>|Ec^zdhk>Qp1Z4%vRlXH3!o+hVK7zL;U^0O-oa>7M@F4mM;?| zelyG3A-efmmUqs}u>Z{4-rf=M{T=vb)q+~H^B1oz_{$oqeyHQh*6DwQvlC0MoN@f^ zR^JH9?vArma@^}VIKb?AP8EIsgB(F@#(9Y)PVx~B5O#8wlY9h7oX>Gqs^<#FjF~<h z&i^@gGB7aR&&|&-V_;y&$t}xBVL$=QFBljYJ}@vcfDl545y1qr(D)!L!Q$8nMg|6^ z?2@9yYz78~n38O;oyaD@*dLI!g9X6^k|-J<Y%hoaVFo7l+?32jP{Em-0=5H5qY(oG z0}n&!5wMVAMoLZ*n8m=r04mgAY$(mRFE=wUje&tdq%0TRAO@8bh@4DHQ7T9cNDs(a zJf%fB3Jl<IW2`GKNy=ehU|3gNQVa`VkeYd*(i@}!q7KS}Q4S0a3?2*&U<?ui=>XBV zvBdxX3=9kwiDk*oj42FU{~t2Qr>8J*t-0x<_WuCG0Y<4tMy3S}49pA)3=C|Jk6bn# zU}Rv+c>Vaa(g6kr27!Xa;sOQ+27VBmfq_AyxTK^QY8l9GHc-98z`)>;j1XsoxSR!~ zjFEwX3rfTIATB87Fo7713=Cj#k>ZkKWOG4g%M_OsCxaA0%me9RfSS#qQ(RJ91(gF) cYzz#{&%yBribDhz0>w~qNpe0&5!4(50FG<tlmGw# literal 0 HcmV?d00001 -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/10321
From: Yuxuan Shui <yshui@codeweavers.com> More specifically the SPS bytes in the avcC atom. Note this only fixes the problem for winedmo but not gstreamer. This is because gstreamer 1) does not parse the SPS, 2) will set pixel-aspect-ratio in caps regardless if a pasp atom is present. So there is no way for us to distinguish between 1) pasp being present and having an aspect ratio of 1:1, and 2) pasp not being present but pixel-aspect-ratio having been set to 1:1 by gstreamer as fallback. And because this is the case, I made sure winedmo is tested separately. In winedmo's case, if pasp is not present, we can set the PixelAspectRatio.Denominator in MFVIDEOFORMAT to 0 to distinguish. --- dlls/mfasfsrcsnk/Makefile.in | 3 +- dlls/mfmp4srcsnk/Makefile.in | 3 +- dlls/mfsrcsnk/Makefile.in | 1 + dlls/mfsrcsnk/media_source.c | 16 ++ dlls/mfsrcsnk/mfsrcsnk_private.h | 1 + dlls/mfsrcsnk/tests/Makefile.in | 2 +- dlls/mfsrcsnk/tests/mfsrcsnk.c | 34 +++- dlls/mfsrcsnk/user_data.c | 320 +++++++++++++++++++++++++++++++ dlls/winedmo/unix_media_type.c | 4 +- 9 files changed, 373 insertions(+), 11 deletions(-) create mode 100644 dlls/mfsrcsnk/user_data.c diff --git a/dlls/mfasfsrcsnk/Makefile.in b/dlls/mfasfsrcsnk/Makefile.in index e2dbc7afb4d..0e7e0f470d7 100644 --- a/dlls/mfasfsrcsnk/Makefile.in +++ b/dlls/mfasfsrcsnk/Makefile.in @@ -8,4 +8,5 @@ SOURCES = \ media_source.c \ mfasfsrcsnk.c \ mfasfsrcsnk.idl \ - mfasfsrcsnk.rc + mfasfsrcsnk.rc \ + user_data.c diff --git a/dlls/mfmp4srcsnk/Makefile.in b/dlls/mfmp4srcsnk/Makefile.in index 5464e1d86e3..860794b8af1 100644 --- a/dlls/mfmp4srcsnk/Makefile.in +++ b/dlls/mfmp4srcsnk/Makefile.in @@ -9,4 +9,5 @@ SOURCES = \ media_source.c \ mfmp4srcsnk.c \ mfmp4srcsnk.idl \ - mfmp4srcsnk.rc + mfmp4srcsnk.rc \ + user_data.c diff --git a/dlls/mfsrcsnk/Makefile.in b/dlls/mfsrcsnk/Makefile.in index 5bef58ca10d..8a6ed16e61f 100644 --- a/dlls/mfsrcsnk/Makefile.in +++ b/dlls/mfsrcsnk/Makefile.in @@ -10,4 +10,5 @@ SOURCES = \ media_source.c \ mfsrcsnk.idl \ mfsrcsnk.rc \ + user_data.c \ wave.c diff --git a/dlls/mfsrcsnk/media_source.c b/dlls/mfsrcsnk/media_source.c index b2ae84002a5..a4f18fdd17a 100644 --- a/dlls/mfsrcsnk/media_source.c +++ b/dlls/mfsrcsnk/media_source.c @@ -1497,6 +1497,7 @@ static const IMFMediaSourceVtbl media_source_vtbl = static HRESULT media_type_from_mf_video_format( const MFVIDEOFORMAT *format, IMFMediaType **media_type ) { HRESULT hr; + GUID subtype; TRACE("format %p, media_type %p\n", format, media_type); @@ -1504,6 +1505,21 @@ static HRESULT media_type_from_mf_video_format( const MFVIDEOFORMAT *format, IMF format->dwSize <= sizeof(*format)) return hr; + if (SUCCEEDED(hr = IMFMediaType_GetGUID(*media_type, &MF_MT_SUBTYPE, &subtype)) && + IsEqualGUID(&subtype, &MFVideoFormat_H264)) + { + UINT8 *user_data; + UINT32 user_data_size; + if (SUCCEEDED(hr = IMFMediaType_GetBlobSize(*media_type, &MF_MT_USER_DATA, &user_data_size)) && + (user_data = malloc(user_data_size))) + { + if (SUCCEEDED(hr = IMFMediaType_GetBlob(*media_type, &MF_MT_USER_DATA, user_data, + user_data_size, &user_data_size))) + mf_media_type_extend_from_user_data(*media_type, user_data, user_data_size); + free(user_data); + } + } + if (FAILED(IMFMediaType_GetItem(*media_type, &MF_MT_VIDEO_ROTATION, NULL))) IMFMediaType_SetUINT32(*media_type, &MF_MT_VIDEO_ROTATION, MFVideoRotationFormat_0); diff --git a/dlls/mfsrcsnk/mfsrcsnk_private.h b/dlls/mfsrcsnk/mfsrcsnk_private.h index 1f0e027f358..41535c1dabf 100644 --- a/dlls/mfsrcsnk/mfsrcsnk_private.h +++ b/dlls/mfsrcsnk/mfsrcsnk_private.h @@ -29,6 +29,7 @@ extern IClassFactory *wave_sink_class_factory; extern IClassFactory *mp3_sink_class_factory; extern IClassFactory *mpeg4_sink_class_factory; +extern void mf_media_type_extend_from_user_data(IMFMediaType *mt, const UINT8 *sps, UINT32 len); static inline HRESULT WINAPI class_factory_QueryInterface(IClassFactory *iface, REFIID riid, void **out) { diff --git a/dlls/mfsrcsnk/tests/Makefile.in b/dlls/mfsrcsnk/tests/Makefile.in index 89889d6f723..f9fc39be487 100644 --- a/dlls/mfsrcsnk/tests/Makefile.in +++ b/dlls/mfsrcsnk/tests/Makefile.in @@ -1,5 +1,5 @@ TESTDLL = mfsrcsnk.dll -IMPORTS = ole32 mfsrcsnk mfplat mf uuid mfuuid +IMPORTS = ole32 mfsrcsnk mfplat mf uuid mfuuid advapi32 SOURCES = \ mfsrcsnk.c \ diff --git a/dlls/mfsrcsnk/tests/mfsrcsnk.c b/dlls/mfsrcsnk/tests/mfsrcsnk.c index d11c681e639..f6d198b6391 100644 --- a/dlls/mfsrcsnk/tests/mfsrcsnk.c +++ b/dlls/mfsrcsnk/tests/mfsrcsnk.c @@ -824,7 +824,7 @@ static void test_thinning(void) IMFMediaSource_Release(source); } -static void test_mp4_extra_metadata(void) +static void test_mp4_extra_metadata(BOOL is_gstreamer) { IMFPresentationDescriptor *presentation_descriptor; IMFStreamDescriptor *stream_descriptor; @@ -866,8 +866,10 @@ static void test_mp4_extra_metadata(void) ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); value64 = 0xdeadbeef; hr = IMFMediaType_GetUINT64(media_type, &MF_MT_PIXEL_ASPECT_RATIO, &value64); - todo_wine ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); - todo_wine ok(value64 == 0x300000004, "Unexpected aspect ratio %#I64x.\n", value64); + todo_if (winetest_platform_is_wine && is_gstreamer) + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + todo_if (winetest_platform_is_wine && is_gstreamer) + ok(value64 == 0x300000004, "Unexpected aspect ratio %#I64x.\n", value64); IMFMediaType_Release(media_type); IMFMediaTypeHandler_Release(type_handler); IMFStreamDescriptor_Release(stream_descriptor); @@ -894,9 +896,11 @@ static void test_mp4_extra_metadata(void) ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); value64 = 0xdeadbeef; hr = IMFMediaType_GetUINT64(media_type, &MF_MT_PIXEL_ASPECT_RATIO, &value64); - todo_wine ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + todo_if (winetest_platform_is_wine && is_gstreamer) + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); /* pasp takes precedence */ - todo_wine ok(value64 == 0xf00000010, "Unexpected aspect ratio: %#I64x.\n", value64); + todo_if (winetest_platform_is_wine && is_gstreamer) + ok(value64 == 0xf00000010, "Unexpected aspect ratio: %#I64x.\n", value64); IMFMediaType_Release(media_type); IMFMediaTypeHandler_Release(type_handler); IMFStreamDescriptor_Release(stream_descriptor); @@ -1005,15 +1009,33 @@ static void test_end_of_presentation(void) START_TEST(mfsrcsnk) { HRESULT hr; + HKEY mf_key; + DWORD disposition; + BOOL one = TRUE; hr = MFStartup(MF_VERSION, MFSTARTUP_FULL); ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); test_wave_sink(); test_thinning(); - test_mp4_extra_metadata(); + test_mp4_extra_metadata(TRUE); test_end_of_presentation(); + if (winetest_platform_is_wine) + { + /* winedmo specific tests. Disable GStreamer first. */ + hr = RegCreateKeyExA(HKEY_CURRENT_USER, "Software\\Wine\\MediaFoundation", 0, NULL, 0, KEY_ALL_ACCESS, NULL, + &mf_key, &disposition); + ok(hr == S_OK, "Unexpected hr %#lx\n", hr); + hr = RegSetValueExA(mf_key, "DisableGstByteStreamHandler", 0, REG_DWORD, (BYTE *)&one, sizeof(one)); + ok(hr == S_OK, "Unexpected hr %#lx\n", hr); + + test_mp4_extra_metadata(FALSE); + + hr = RegDeleteValueA(mf_key, "DisableGstByteStreamHandler"); + ok(hr == S_OK, "Unexpected hr %#lx\n", hr); + } + hr = MFShutdown(); ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); } diff --git a/dlls/mfsrcsnk/user_data.c b/dlls/mfsrcsnk/user_data.c new file mode 100644 index 00000000000..c3324ecd0c7 --- /dev/null +++ b/dlls/mfsrcsnk/user_data.c @@ -0,0 +1,320 @@ +#define COBJMACROS +#include "mfsrcsnk_private.h" + +WINE_DEFAULT_DEBUG_CHANNEL(mfplat); + +struct nal_reader +{ + /* Pointer to the slice of bytes to read. */ + const UINT8 *data; + /* Number of bytes in data. */ + UINT32 len; + /* Accumulated bits from data, no processing of emulation prevention bytes. */ + UINT32 raw_bits; + /* Accumulated bits from data, excluding the first_byte */ + UINT32 cache; + /* Number of valid bits in cache. */ + UINT8 num_bits_in_cache; +}; + +/* Read from data until there are `num_bits` in cache */ +static BOOLEAN nal_reader_fill(struct nal_reader *reader, UINT8 num_bits) +{ + while (reader->num_bits_in_cache < num_bits) + { + UINT8 byte; + if (!reader->len) return FALSE; + byte = *reader->data; + reader->data++; + + reader->raw_bits = (reader->raw_bits << 8) | byte; + if ((reader->raw_bits & 0xffffff) == 0x3) { + /* EPB (emulation prevention byte), ignore. */ + continue; + } + + reader->cache = (reader->cache << 8) | byte; + reader->num_bits_in_cache += 8; + } + return TRUE; +} + +static BOOLEAN nal_reader_skip(struct nal_reader *reader, UINT8 num_bits) +{ + if (!nal_reader_fill(reader, num_bits)) return FALSE; + reader->num_bits_in_cache -= num_bits; + return TRUE; +} + +static BOOLEAN nal_reader_read_bits(struct nal_reader *reader, UINT8 num_bits, UINT32 *out) +{ + UINT32 remaining, value; + if (!nal_reader_fill(reader, num_bits)) return FALSE; + + remaining = reader->num_bits_in_cache - num_bits; + value = reader->cache >> remaining; + if (num_bits < 32) { + value &= (1 << num_bits) - 1; + } + reader->num_bits_in_cache = remaining; + *out = value; + return TRUE; +} + +static BOOLEAN nal_reader_read_ue(struct nal_reader *reader, UINT32 *out) +{ + /* https://en.wikipedia.org/wiki/Exponential-Golomb_coding */ + UINT8 shift = 0; + UINT32 value; + while (TRUE) + { + UINT32 tmp; + if (!nal_reader_read_bits(reader, 1, &tmp)) return FALSE; + if (tmp) break; + shift++; + } + + if (!nal_reader_read_bits(reader, shift, &value)) return FALSE; + *out = (1 << shift) - 1; + *out += value; + return TRUE; +} + +static BOOLEAN nal_reader_skip_ue(struct nal_reader *reader) +{ + UINT32 tmp; + return nal_reader_read_ue(reader, &tmp); +} + +static BOOLEAN nal_reader_skip_n_ues(struct nal_reader *reader, UINT32 num) +{ + while (num--) + if (!nal_reader_skip_ue(reader)) + return FALSE; + return TRUE; +} + +static void nal_reader_init(struct nal_reader *reader, const UINT8 *data, UINT32 len) +{ + reader->data = data; + reader->len = len; + /* raw_bits can't be 0, otherwise if the first byte is 0x3 we will treat it as + * an EPB */ + reader->raw_bits = 0xff; + reader->cache = 0; + reader->num_bits_in_cache = 0; +} + +static BOOLEAN skip_scaling_matrices(struct nal_reader *reader, UINT32 num_matrices) +{ + UINT32 i; + for (i = 0; i < num_matrices; i++) + { + UINT32 matrix_present, matrix_size = i < 6 ? 16 : 64, j, last = 8, next; + if (!nal_reader_read_bits(reader, 1, &matrix_present)) return FALSE; + if (!matrix_present) continue; + for (j = 0; j < matrix_size; j++) + { + UINT32 tmp; + INT32 delta; + if (!nal_reader_read_ue(reader, &tmp)) return FALSE; + delta = tmp % 2 ? (INT32)(tmp / 2) + 1 : -(INT32)(tmp / 2); + next = last + delta; + if (!next) break; + last = next; + } + } + return TRUE; +} + +/* Table E-1 - Meaning of sample aspect ratio indicator (1..16) */ +static const struct +{ + UINT32 width; + UINT32 height; +} aspect_ratios[17] = +{ + {0, 0}, + {1, 1}, + {12, 11}, + {10, 11}, + {16, 11}, + {40, 33}, + {24, 11}, + {20, 11}, + {32, 11}, + {80, 33}, + {18, 11}, + {15, 11}, + {64, 33}, + {160, 99}, + {4, 3}, + {3, 2}, + {2, 1} +}; + +static UINT32 scan_for_start_code(const UINT8 *data, UINT32 len, UINT32 *start_code_len) +{ + static const UINT8 start_code3[] = {0x00, 0x00, 0x01}; + static const UINT8 start_code4[] = {0x00, 0x00, 0x00, 0x01}; + UINT32 offset = 0; + while (offset + 3 <= len) + { + if (!memcmp(data + offset, start_code3, 3)) + { + *start_code_len = 3; + return offset; + } + if (offset + 4 <= len && !memcmp(data + offset, start_code4, 4)) + { + *start_code_len = 4; + return offset; + } + offset++; + } + return len; +} + +/* Extend a IMFMediaType with information from IMFMediaType user data. + * Right now, only h264 user data, i.e. NAL units are handled. And we only use the pixel aspect + * ratio information from an SPS (Sequence Parameter Set) NAL unit. */ +void mf_media_type_extend_from_user_data(IMFMediaType *mt, const UINT8 *data, UINT32 len) +{ + UINT32 profile_idc, start_code_len; + struct nal_reader reader; + UINT32 tmp, i, offset; + UINT64 par; + + TRACE("user_data: "); + for (i = 0; i < len; i++) TRACE("%02x", (unsigned char)data[i]); + TRACE("\n"); + + if (SUCCEEDED(IMFMediaType_GetUINT64(mt, &MF_MT_PIXEL_ASPECT_RATIO, &par))) + { + /* This media type already has a aspect ratio, presumably from the container, e.g. the pasp + * atom in an mp4 file. */ + TRACE("Already has aspect ratio\n"); + return; + } + + /* Check for start code */ + offset = scan_for_start_code(data, len, &start_code_len); + if (offset != 0) + { + TRACE("No start code found\n"); + return; + } + data += start_code_len; + len -= start_code_len; + + /* + * Scan for a SPS NAL unit. Check if we get a SPS NAL header, + * if not, scan for the next start code. + * + * 0x67 is the NAL header for SPS. + * 0x67 == 0b01100111 + * + * 0 - forbidden_zero_bit (1 bit) + * 11 - nal_ref_idc (2 bits), 3 for SPS + * 00111 - nal_unit_type (5 bits), 7 for SPS + */ + while (*data != 0x67) + { + offset = scan_for_start_code(data, len, &start_code_len); + if (offset >= len) + { + TRACE("No SPS found.\n"); + return; + } + /* skip to the start of the next NAL unit, skip over the start code. */ + data += offset + start_code_len; + len -= offset + start_code_len; + } + + nal_reader_init(&reader, data, len); + nal_reader_skip(&reader, 8); /* skip nal header */ + + /* We only care about the VUI (Video Usability Information) for now, so we skip over most of + * the data. */ + if (!nal_reader_read_bits(&reader, 8, &profile_idc) || !nal_reader_skip(&reader, 16) || + !nal_reader_skip_ue(&reader)) + return; + + if (profile_idc == 100 || profile_idc == 110 || profile_idc == 122 || profile_idc == 244 || + profile_idc ==44 || profile_idc == 83 || profile_idc == 86 || profile_idc == 118 || + profile_idc == 128 || profile_idc == 138 || profile_idc == 139 || profile_idc == 134 || + profile_idc == 135) + { + /* High profile */ + UINT32 chroma_format_id, scaling_matrix_present; + if (!nal_reader_read_ue(&reader, &chroma_format_id)) return; + if (chroma_format_id == 3 && !nal_reader_skip(&reader, 1)) return; + if (!nal_reader_skip_n_ues(&reader, 2) || !nal_reader_skip(&reader, 1) || + !nal_reader_read_bits(&reader, 1, &scaling_matrix_present))\ + return; + if (scaling_matrix_present) { + UINT32 num_matrices = chroma_format_id == 3 ? 8 : 12; + if (!skip_scaling_matrices(&reader, num_matrices)) return; + } + } + + if (!nal_reader_skip_ue(&reader) || !nal_reader_read_ue(&reader, &tmp)) return; + + /* pic_order_cnt_type */ + if (!tmp) + { + if (!nal_reader_skip_ue(&reader)) return; + } + else if (tmp == 1) + { + UINT32 num_ref_frames; + if (!nal_reader_skip(&reader, 1) || !nal_reader_skip_n_ues(&reader, 2) || + !nal_reader_read_ue(&reader, &num_ref_frames)) + return; + while (num_ref_frames--) if (!nal_reader_skip_ue(&reader)) return; + } + if (!nal_reader_skip_ue(&reader) || !nal_reader_skip(&reader, 1) || + !nal_reader_skip_n_ues(&reader, 2) || !nal_reader_read_bits(&reader, 1, &tmp)) + return; + + /* frame_mbs_only_flag */ + if (!tmp && !nal_reader_skip(&reader, 1)) return; + if (!nal_reader_skip(&reader, 1) || !nal_reader_read_bits(&reader, 1, &tmp)) return; + + /* frame_cropping_flag */ + if (tmp && !nal_reader_skip_n_ues(&reader, 4)) return; + + /* Finally, this is the VUI */ + if (!nal_reader_read_bits(&reader, 1, &tmp)) return; + if (tmp) + { + UINT32 aspect_ratio_present_flag; + if (!nal_reader_read_bits(&reader, 1, &aspect_ratio_present_flag)) return; + if (aspect_ratio_present_flag) + { + UINT32 aspect_ratio_idc; + if (!nal_reader_read_bits(&reader, 8, &aspect_ratio_idc)) return; + if (aspect_ratio_idc == 255) + { + // Extended aspect ratio + UINT32 width, height; + if (!nal_reader_read_bits(&reader, 16, &width) || + !nal_reader_read_bits(&reader, 16, &height)) + return; + par = (((UINT64)width) << 32) | height; + } + else if (aspect_ratio_idc < 17) + par = (((UINT64)aspect_ratios[aspect_ratio_idc].width) << 32) | + aspect_ratios[aspect_ratio_idc].height; + else + { + WARN("Invalid aspect_ratio_idc: %u\n", aspect_ratio_idc); + return; + } + + if (FAILED(IMFMediaType_SetUINT64(mt, &MF_MT_PIXEL_ASPECT_RATIO, par))) + WARN("Failed to set aspect ratio.\n"); + } + } + else TRACE("VUI not found in SPS.\n"); +} diff --git a/dlls/winedmo/unix_media_type.c b/dlls/winedmo/unix_media_type.c index fe3060eaea7..0d6f03f082e 100644 --- a/dlls/winedmo/unix_media_type.c +++ b/dlls/winedmo/unix_media_type.c @@ -262,8 +262,8 @@ static void mf_video_format_init( const AVCodecParameters *params, MFVIDEOFORMAT } else { - format->videoInfo.PixelAspectRatio.Numerator = 1; - format->videoInfo.PixelAspectRatio.Denominator = 1; + format->videoInfo.PixelAspectRatio.Numerator = 0; + format->videoInfo.PixelAspectRatio.Denominator = 0; } if (fps->num && fps->den) -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/10321
slight simplification of NAL bit stream reading. -- https://gitlab.winehq.org/wine/wine/-/merge_requests/10321#note_132254
This looks like something that should be fixed in FFMpeg? We use stream->sample_aspect_ratio to get SAR info, I would assume that it would be parsed from the codec data and I'm quite surprised that it's not already. We might be misusing the FFMpeg API instead, and maybe the information is available elsewhere. It's also possible that it needs `avformat_find_stream_info` to be called before it is filled. We don't always call it because it can be costly, and demuxer initialization needed to be as quick as possible, but maybe we can check if SAR info is missing and decide to call it in that case (or somehow delay until we *need* SAR info). -- https://gitlab.winehq.org/wine/wine/-/merge_requests/10321#note_133369
On Wed Mar 25 12:36:18 2026 +0000, Rémi Bernon wrote:
This looks like something that should be fixed in FFMpeg? We use stream->sample_aspect_ratio to get SAR info, I would assume that it would be parsed from the codec data and I'm quite surprised that it's not already. We might be misusing the FFMpeg API instead, and maybe the information is available elsewhere. It's also possible that it needs `avformat_find_stream_info` to be called before it is filled. We don't always call it because it can be costly, and demuxer initialization needed to be as quick as possible, but maybe we can check if SAR info is missing and decide to call it in that case (or somehow delay until we *need* SAR info). hmm, ok. I guess I can write an ffmpeg test program and dig around to see if this information is available anywhere. It might be the case that this information is only available after decoding a frame.
-- https://gitlab.winehq.org/wine/wine/-/merge_requests/10321#note_133704
On Wed Mar 25 12:36:18 2026 +0000, Yuxuan Shui wrote:
hmm, ok. I guess I can write an ffmpeg test program and dig around to see if this information is available anywhere. It might be the case that this information is only available after decoding a frame. ok, yes. this info is available in `codecpar` after `avformat_find_stream_info`.
-- https://gitlab.winehq.org/wine/wine/-/merge_requests/10321#note_133705
On Wed Mar 25 13:08:44 2026 +0000, Yuxuan Shui wrote:
ok, yes. this info is available in `codecpar` after `avformat_find_stream_info`. right. `avformat_find_stream_info` doesn't get the sar until it gets to `try_decode_frame`. but actually, SAR is available way earlier, there is just no API for getting it.
`ff_h264_decode_extradata` is where the SPS gets parsed, which is called by, for example, `h264_decode_init`, `h264_parse`, etc. One option is we just call `ff_h264_decode_extradata` ourselves. We can drop my SPS parser implementation if we do that. How does that sound? -- https://gitlab.winehq.org/wine/wine/-/merge_requests/10321#note_133714
participants (3)
-
Rémi Bernon -
Yuxuan Shui
-
Yuxuan Shui (＠yshui)