[PATCH 0/1] MR8887: mf/tests: Test H.264 sink media type height alignment.
H.264 uses a 16-pixel alignment, and the stream sink media type should have the aligned height after the session has started. -- https://gitlab.winehq.org/wine/wine/-/merge_requests/8887
From: Conor McCarthy <cmccarthy(a)codeweavers.com> H.264 uses a 16-pixel alignment, and the stream sink media type should have the aligned height after the session has started. --- dlls/mf/tests/mf.c | 115 +++++++++++++++++++++++++++++++ dlls/mf/tests/resource.rc | 9 +++ dlls/mf/tests/test-unaligned.mp4 | Bin 0 -> 3988 bytes 3 files changed, 124 insertions(+) create mode 100644 dlls/mf/tests/test-unaligned.mp4 diff --git a/dlls/mf/tests/mf.c b/dlls/mf/tests/mf.c index a02f2859f91..1683c73f2f7 100644 --- a/dlls/mf/tests/mf.c +++ b/dlls/mf/tests/mf.c @@ -8040,6 +8040,120 @@ static void test_media_session_seek(void) ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); } +static void test_h264_output_alignment(void) +{ + media_type_desc video_nv12_desc = + { + ATTR_GUID(MF_MT_MAJOR_TYPE, MFMediaType_Video), + ATTR_GUID(MF_MT_SUBTYPE, MFVideoFormat_NV12), + }; + struct test_grabber_callback *grabber_callback; + IMFTopology *topology, *resolved_topology; + IMFMediaTypeHandler *handler; + IMFActivate *sink_activate; + IMFTopoLoader *topo_loader; + IMFStreamSink *stream_sink; + IMFAsyncCallback *callback; + IMFMediaType *output_type; + IMFMediaSession *session; + IMFMediaSink *media_sink; + IMFMediaSource *source; + PROPVARIANT propvar; + UINT64 frame_size; + HRESULT hr; + + hr = MFStartup(MF_VERSION, MFSTARTUP_FULL); + ok(hr == S_OK, "Failed to start up, hr %#lx.\n", hr); + + if (!(source = create_media_source(L"test-unaligned.mp4", L"video/mp4"))) + { + todo_wine /* Gitlab CI Debian runner */ + win_skip("MP4 media source is not supported, skipping tests.\n"); + MFShutdown(); + return; + } + + grabber_callback = impl_from_IMFSampleGrabberSinkCallback(create_test_grabber_callback()); + hr = MFCreateMediaType(&output_type); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + init_media_type(output_type, video_nv12_desc, -1); + hr = MFCreateSampleGrabberSinkActivate(output_type, &grabber_callback->IMFSampleGrabberSinkCallback_iface, &sink_activate); + ok(hr == S_OK, "Failed to create grabber sink, hr %#lx.\n", hr); + IMFMediaType_Release(output_type); + + IMFActivate_ActivateObject(sink_activate, &IID_IMFMediaSink, (void **)&media_sink); + hr = IMFMediaSink_GetStreamSinkByIndex(media_sink, 0, &stream_sink); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + IMFActivate_Release(sink_activate); + topology = create_test_topology_unk(source, (IUnknown *)stream_sink, NULL, NULL); + hr = MFCreateTopoLoader(&topo_loader); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + hr = IMFTopoLoader_Load(topo_loader, topology, &resolved_topology, NULL); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + IMFTopology_Release(topology); + IMFTopoLoader_Release(topo_loader); + + hr = IMFStreamSink_GetMediaTypeHandler(stream_sink, &handler); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + hr = IMFMediaTypeHandler_GetCurrentMediaType(handler, &output_type); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + hr = IMFMediaType_GetUINT64(output_type, &MF_MT_FRAME_SIZE, &frame_size); + ok(hr == MF_E_ATTRIBUTENOTFOUND, "Unexpected hr %#lx.\n", hr); + IMFMediaType_Release(output_type); + + hr = MFCreateMediaSession(NULL, &session); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + hr = IMFMediaSession_SetTopology(session, 0, resolved_topology); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + IMFTopology_Release(resolved_topology); + callback = create_test_callback(TRUE); + hr = wait_media_event(session, callback, MESessionTopologySet, 1000, &propvar); + + hr = IMFMediaTypeHandler_GetCurrentMediaType(handler, &output_type); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + hr = IMFMediaType_GetUINT64(output_type, &MF_MT_FRAME_SIZE, &frame_size); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + ok((UINT32)frame_size == 72, "Unexpected frame height %u\n", (UINT32)frame_size); + IMFMediaType_Release(output_type); + + propvar.vt = VT_EMPTY; + hr = IMFMediaSession_Start(session, &GUID_NULL, &propvar); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + hr = wait_media_event(session, callback, MESessionStarted, 5000, &propvar); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + + hr = IMFMediaTypeHandler_GetCurrentMediaType(handler, &output_type); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + hr = IMFMediaType_GetUINT64(output_type, &MF_MT_FRAME_SIZE, &frame_size); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + todo_wine + ok((UINT32)frame_size == 80, "Unexpected frame height %u\n", (UINT32)frame_size); + IMFMediaType_Release(output_type); + + hr = IMFMediaSession_Stop(session); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + hr = IMFMediaSession_Close(session); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + hr = wait_media_event(session, callback, MESessionClosed, 1000, &propvar); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + + hr = IMFMediaSource_Shutdown(source); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + hr = IMFMediaSession_Shutdown(session); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + + IMFMediaTypeHandler_Release(handler); + IMFAsyncCallback_Release(callback); + IMFMediaSession_Release(session); + IMFStreamSink_Release(stream_sink); + IMFMediaSink_Release(media_sink); + IMFSampleGrabberSinkCallback_Release(&grabber_callback->IMFSampleGrabberSinkCallback_iface); + IMFMediaSource_Release(source); + + hr = MFShutdown(); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); +} + START_TEST(mf) { init_functions(); @@ -8078,4 +8192,5 @@ START_TEST(mf) test_media_session_source_shutdown(); test_media_session_thinning(); test_media_session_seek(); + test_h264_output_alignment(); } diff --git a/dlls/mf/tests/resource.rc b/dlls/mf/tests/resource.rc index dff5719dd3b..2a69a842ce8 100644 --- a/dlls/mf/tests/resource.rc +++ b/dlls/mf/tests/resource.rc @@ -185,3 +185,12 @@ rgb24frame.bmp RCDATA rgb24frame.bmp */ /* @makedep: test.mp4 */ test.mp4 RCDATA test.mp4 + +/* Generated with: + * gst-launch-1.0 videotestsrc num-buffers=60 pattern=smpte100 ! \ + * video/x-raw,format=I420,width=72,height=64,framerate=30000/1001 ! \ + * videoflip method=clockwise ! videoconvert ! \ + * x264enc ! mp4mux ! filesink location=dlls/mf/tests/test-unaligned.mp4 + */ +/* @makedep: test-unaligned.mp4 */ +test-unaligned.mp4 RCDATA test-unaligned.mp4 diff --git a/dlls/mf/tests/test-unaligned.mp4 b/dlls/mf/tests/test-unaligned.mp4 new file mode 100644 index 0000000000000000000000000000000000000000..5920da40daaf55dc69a5a02577891c70906fed94 GIT binary patch literal 3988 zcmZQzU{FXasVvAXFfn3aU|;~zxdkSMnZ^0JnZ@}aF^;sN)KmrrPSxC$#1aMu1}07c z1_lP{^b`h>H8)-KpB{*6W(a)Zj*U|{^xz)))7-X(IHfq{WF<Mo?OIt&a<-`H6H|NnHy z_2u4YH$Apb)48E=Q~BNd3L`TU1zm;Y{GwC^Lo*YFB4a}X3k74tR71-&O9fp84?QC@ z6Mf$RS9e_#1;;RFh2;E{)MN!+1?T*N%AC}+5(Ogz17lqy10!PvU4(a)L2k^(Dz{qpj1 zy|T=d)cl;pJiYv)bbXL5dKo3TISRT8`2{7J`FX`w3dxB{iOIHx3Pq`Dw#Eu6sYyBc z$=SAsRt8oE3W<4%IhDn!wgwf(Rt6P(a)hQ<oHskRxZ6$-_rNx7-E<_ZPHm9~Z;GQKFq z)=<yDz{)_+z(654vm!Mm9%Q1SLT+k&QDR<ts;!}!LUKk?er{rXZmO-JLP=3-PEKaA zt)YTNg+)qoiLIeRa$&BmfkH}ZVoFthUaGB;p^l-ULRw;RNqj+Zc4h%c1;l{Dg82Nj zwBpnfTU{fCl8mC%#FS!NBZZv&{OrVx)Wno{n3$nLaZYA(Dnh_OA+HE*M{;IvVhP9{ znRz9tMLCJdsVTMw3Q0MoMTwR1$@#eji6yoM3d#9-#U(|FnR%%x(a)tJugMTsCaX+?>- zsl~R&3Q6$=l|_lUnJKnL3Q6&aDTxIjUnIpRWhNGbEXgcN1^cT!H8VY<B+1rLA-^Cs zFFrlLz}7$kB3NK+q>!CjnVDB&Yh-B&Wya(a)b=GhuqDikND=A|Z=me`sYD1dDT1!+-g zafYpdLQ!%&+`Az8qGa3Tq#}jfq>`dkkhhXDONu}qG%_%;Pyn$?@^e5jq)-U*oUMVL znSnxK0mvdSomgROW~oqEP+XE)U~8gKmQ)s>n^*zTXA3p~B#=~^R-9RtYHMg(a)WTB9k zm=m8-lwzBgpO>nTmYI{23JTZEg7~7ulFWQtLp>7%g~UQzLn~0KU}#`qU}Q<{Xkn20 z|K`7+{kA6+Qd^2I`~RD(IQt;`!_!ZS`W{3tl$(9;*`dYfgPxbnlh+ChU7Q+gb(=+I z{(`-Sx|+GYWg_lec%)u?W%Y`b1Jit*Cg*>g_O<)J{Kj-=af=s=mSoHP-^1=C>K-zG z?xyvgJlpH1sxN-E)OeBDHirG{&8q#AGOG#=G{35ZD(a)-(Bae+l8>hs^Vsqd=mnQkPl ze%hF867*ujlUJ(>el5Jw&lM#!jpNT$ncp^=brbx$zothi8rom{e)i~|I<I;6{_oNM z=CJc~rD<}2rv&rc69Fc@)%@4zNoCzVWO%nLM<MPTgFxq*hf0^9o(a)1C^Qr(r&kzqLV zxBjQ)%ihjW4hNSP1`G(a)g+>WzUavazHyR|Gi11uQ8z`(%iIM1n~=|4l9BSMIAUSf%3 zKEt#IxDW^9yyPrLd1i!Rf{wE?JXbho%=F<f{SPuS6Rw!sabBCNppMvohG~s(K(a)P@w zOG_N>87tu;oQ(69XF2*ayo6W-a-pE(tTmo19CP4?BHYRCIPa9JsE)*c1`|bOBVMAp zl#_AZ>nw!hp{9O7F%{XV^ZBrtI$sxTD#G=gjPvy|Or33lVk)v*=LcakwGeD7#Jk|I zXPjS*X=(+EsVHup1~pXyDFo(k1)GZE*6m<ZAwrN4n7s$ZR1~+~f|`mPuJeC_O+|6* zZ%k7kpqPs6)_HPJQ<2>|&lbbfdG;8l&Qd`!71^!xoS>#6yA>m~a)MK9J}mh`LI9jx z5h)ABt!>y$T?%$9BoHC71xl~*YymZOG`+&otMUJT*Oo>jhwFR_((7($8qod!Z>!ec z2k->X?KuCYtEi4RT8TCP7dRO;Ffg#W=jP{^F)%RX<d$WmfErX!?y&TLFatxW0RzLU zISh;pAOw<vU`7Z9C(-y0a0#poCI$x9X(dI8*$fN}F(uhxJJ~_(a)fiMHys#kL`bfejW z>L>;Vu)QF{1FTaeHKnAOfq_9LHK!QV2m}c-g8jq5ehH$NWovFqW+JFTnVSN32Ls3r zAPn<+=n;@M2HlL5oFXubfq?<kV1uzibQq`s=aHC~l9LKj!;+AjnU}`Ez#vkV3pNzl z*(xazIhmBARFE2wevmv*X;BU+=QA=euqYIlB;_zLFkC1uDF#OfG*AvCmL(g4G(h=K zCX{ku(a)L&LW9E3qaAX7m!ZY=TtKLZ1Uabj7rGh+&a$p43UTgZ&?1~P+4Qb`dcL_tnw zPyxlHaB_Z5QC(a)OR1p^xxGB7X*6eJcGK-~&*jYM$?I6^=!0`YAa7?@vz!l}5V7!*-3 zagdx$aY=D9NB~K{4TR0eQe0A81(gF)HVh0bpwdN(a)fq_97iiJS!eFg>wkQj&!5|@VZ zL2M8n!hFUT$%wGDhJ*|IB?v9t3!!0rP!NHVJ2<Qv85l<82Y2}JF98+ah`f=U4~{`d z!3av{k)<gmph7D!Hx*2Sq!nQWRBlF2QEo~ms2~D~!?8*xq!g4{S&|PjP$VT0tQnNq V7(l&F1_lOOsKKVB7|a1O7yw{rbdCT3 literal 0 HcmV?d00001 -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/8887
It looks like tests are needed for the transform output type too, because it's unclear if that changes at the same time as the sink input. Also I wasn't able to reproduce the Window 10 crash locally, but the CI artifacts contain very little info. Is there a way to get more? -- https://gitlab.winehq.org/wine/wine/-/merge_requests/8887#note_114743
Nikolay Sivov (@nsivov) commented about dlls/mf/tests/mf.c:
+ ok((UINT32)frame_size == 72, "Unexpected frame height %u\n", (UINT32)frame_size); + IMFMediaType_Release(output_type); + + propvar.vt = VT_EMPTY; + hr = IMFMediaSession_Start(session, &GUID_NULL, &propvar); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + hr = wait_media_event(session, callback, MESessionStarted, 5000, &propvar); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + + hr = IMFMediaTypeHandler_GetCurrentMediaType(handler, &output_type); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + hr = IMFMediaType_GetUINT64(output_type, &MF_MT_FRAME_SIZE, &frame_size); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + todo_wine + ok((UINT32)frame_size == 80, "Unexpected frame height %u\n", (UINT32)frame_size); + IMFMediaType_Release(output_type); In my opinion the way this is tested is too high level. It's of course fine to test what topology resolution produces, and initial media source types. But if you only compare before and after session state change, there is no way to see from such test where this type change happened. Does it mean source type changes once source is started, or H.264 decoder changes types on its own or in response to source type change?
Does this break application assumptions on output buffer sizes? -- https://gitlab.winehq.org/wine/wine/-/merge_requests/8887#note_114865
On Thu Sep 4 11:07:06 2025 +0000, Nikolay Sivov wrote:
In my opinion the way this is tested is too high level. It's of course fine to test what topology resolution produces, and initial media source types. But if you only compare before and after session state change, there is no way to see from such test where this type change happened. Does it mean source type changes once source is started, or H.264 decoder changes types on its own or in response to source type change? Does this break application assumptions on output buffer sizes? By wrapping the transform, sink and sink media type handler I found the type switch occurs after the transform has processed some samples. It always occurs before the first sample is sent to the sink. The session engine calls `GetOutputAvailableType()` on the transform, and then with the new type it calls `SetOutputType()` on the transform and `SetCurrentMediaType()` on the sink type handler. This happens for H.264 even for aligned sizes, but it is not done for WMV.
It may be possible to define the sequence of events more completely by wrapping the `IMFMediaSession`, but I think that's unnecessary and would be complicated. Applications must depend for their sequencing on the same events used in this test, so it's unlikely any will break on a subtle difference. It should suffice to update the type before the sink receives the first sample and also before `MESessionStarted`. I also tested `MESessionTopologyStatus/MF_TOPOSTATUS_STARTED_SOURCE` which showed the change occurs after that. That's the last event before `MESessionStarted`. I will be adding that test to the MR. I'm not planning to include the wrapped transform, since we need only check for sample delivery to the grabber. I'll add a test for that. Do you have suggestions for other tests? -- https://gitlab.winehq.org/wine/wine/-/merge_requests/8887#note_114983
On Fri Sep 5 10:55:55 2025 +0000, Conor McCarthy wrote:
By wrapping the transform, sink and sink media type handler I found the type switch occurs after the transform has processed some samples. It always occurs before the first sample is sent to the sink. The session engine calls `GetOutputAvailableType()` on the transform, and then with the new type it calls `SetOutputType()` on the transform and `SetCurrentMediaType()` on the sink type handler. This happens for H.264 even for aligned sizes, but it is not done for WMV. It may be possible to define the sequence of events more completely by wrapping the `IMFMediaSession`, but I think that's unnecessary and would be complicated. Applications must depend for their sequencing on the same events used in this test, so it's unlikely any will break on a subtle difference. It should suffice to update the type before the sink receives the first sample and also before `MESessionStarted`. I also tested `MESessionTopologyStatus/MF_TOPOSTATUS_STARTED_SOURCE` which showed the change occurs after that. That's the last event before `MESessionStarted`. I will be adding that test to the MR. I'm not planning to include the wrapped transform, since we need only check for sample delivery to the grabber. I'll add a test for that. Do you have suggestions for other tests? If decoder is doing this type adjustment, could we test decoder transform directly, without any wrappers?
-- https://gitlab.winehq.org/wine/wine/-/merge_requests/8887#note_114984
On Fri Sep 5 11:11:58 2025 +0000, Nikolay Sivov wrote:
If decoder is doing this type adjustment, could we test decoder transform directly, without any wrappers? I intend to add `GetOutputType()` tests on the transform, yes, and probably `GetInputType()` is relevant too.
Does this break application assumptions on output buffer sizes?
I guess apps must assume the buffer is aligned. Satisfactory is the only game I know of which checks the sink input type, and video display is bugged as a result. -- https://gitlab.winehq.org/wine/wine/-/merge_requests/8887#note_114986
participants (3)
-
Conor McCarthy -
Conor McCarthy (＠cmccarthy)
-
Nikolay Sivov (＠nsivov)