[PATCH 0/10] MR2893: winegstreamer: Implement drain, flush and low-latency for the H264 decoder.
From: Rémi Bernon rbernon@codeweavers.com
This will eventually fallback to vaapih264dec and similar decoders if VA-API plugins are indeed available. --- dlls/winegstreamer/unixlib.c | 8 ++++++++ 1 file changed, 8 insertions(+)
diff --git a/dlls/winegstreamer/unixlib.c b/dlls/winegstreamer/unixlib.c index 6ffd41e9712..6cdcfa6758a 100644 --- a/dlls/winegstreamer/unixlib.c +++ b/dlls/winegstreamer/unixlib.c @@ -101,6 +101,14 @@ GstElement *find_element(GstElementFactoryListType type, GstCaps *src_caps, GstC for (tmp = transforms; tmp != NULL && element == NULL; tmp = tmp->next) { name = gst_plugin_feature_get_name(GST_PLUGIN_FEATURE(tmp->data)); + + if (!strcmp(name, "vaapidecodebin")) + { + /* vaapidecodebin adds too much abstraction and wg_transform works best with lower-level elements. */ + GST_WARNING("Ignoring vaapidecodebin decoder."); + continue; + } + if (!(element = gst_element_factory_create(GST_ELEMENT_FACTORY(tmp->data), NULL))) GST_WARNING("Failed to create %s element.", name); }
From: Rémi Bernon rbernon@codeweavers.com
--- dlls/mf/tests/h264-concat-streams.bin | Bin 0 -> 4552 bytes dlls/mf/tests/mf_test.h | 1 + dlls/mf/tests/resource.rc | 15 +++ dlls/mf/tests/transform.c | 173 ++++++++++++++++++++++++-- 4 files changed, 176 insertions(+), 13 deletions(-) create mode 100644 dlls/mf/tests/h264-concat-streams.bin
diff --git a/dlls/mf/tests/h264-concat-streams.bin b/dlls/mf/tests/h264-concat-streams.bin new file mode 100644 index 0000000000000000000000000000000000000000..dc356da7c485365a70bd65171593f471bd12d9e7 GIT binary patch literal 4552 zcmeH~YjD(56vwl43a!%0L$oqeh_;}EePka(9|l~pP+n46-stRPv)SFy>?Tb@y9J~O ziZ2{srXsX-7nJg5tRe#nh*XA_hmPW7T7+pGMzjK2iUU%h>$zE)CRsi(ew8oj&AIpf z&i~$f_U2y<!%Xr0FnD@g*u%S;$Bhc6nCAEGn=lc>`kcm0KO7nIR8bo0_g=j82{cT- zW5eN&vw3ZYE;Q%wTbtH|H>dphSHLpThNt5+FEY3pnxd7+eg=0^3}Yh*=*Z820hv<@ z^2VpzaQn=$Fw(`)&@q-5$~nd@;}(LjrdtS$1%h5#7IHE(%gV|!N?8}fbCf@W7d@FM zONLkWaWIA#WR~|!IXF!@DVj9nBI72lxQlUeJY7thbBG)Qr~DLGE-@q#u;vf}v)PLK z7}CoGaA|=PTBC45Du=1?bcim}oI$`Oc;G%Zz_=WUXoeXMk@9;O(mWEUy&~_U95B_4 z%Ob;ZtOTvBK$eS^p+T4UNCG5Ku7$jxAuZ-Kvl(|&lI#$qVpc#SB7hRX!MoiOBa`VC zT=t4ELjv&}&lgi(cyV;c%(%p{G-&A#5V&7da?q@gl93M9FEb)X!61k`xdkFs?x1;} zK*{h<!xqUR#rgpO14Rmraf_6XLHV2xp&Z(*i?l$Sa!~?Wh120=DG4QEMMhay8N+(K zvJ?7vf$=*$ya2tCpa4^gnQ~y0mh9|1O^1*5!z(V)jGv(w$fS)xGDQk)smMrPm@d)| zb-Actq)FN-;yx$rD_WM5h4HXJ3t`K`QA_4IbV7IuS{zAajD)6u5-ESF068)n0+P%K zqzx~1mO6Y?pof40&IN9XT?lZCB@2hjaCqU+{cuvao8>qL?JFz5VMAd-CeTEcKpml~ zz?MScGjG0z4Gv$r`RbS@<DN~Jn|*r#*Ll&_cBG@>>53aIFEu8gt~l3!wWnzJ_e-nh zZw=c%ZOd8oP5vW?^1j^k-l!8V)t>*X@a#X6Owxv=#qQjLi%Uj?(^ld6m2K^76SGdn z)QmcPyKaDR?C<?uNpni)9xbZmi`VptJ>^-qEg3s6_@wRUn&$j+VgL4&Juk%G`thT6 zAOBEU+Vs$#uksF$?p%ImZ)Z#Ag^gF=wH|CrE>5l|&*wK3^lOlI%-eQ7_Hp#{jVIt? zuS(_YA9bDCGJANd5-dQ$;PZiHVJx9pjhKRzZ0E70c1?tyk0Mu;NCf4bu+8pWKRNE1 zumb!=9c&M-$m^e$6vq0bt1(lsRzXGpD6J{DRY69QM3B1x$@L&x)#>)&iM&Bk$Trzt zfh9DlqfEhz^Lru@WV;@zQ5i~%MuyTAq-HBZnMP!_O^?*53>Dr5Sz=TflA@3SJyNT3 z?H#0Y3CAlt7up-8a-V{X=vEe9eNd0osyuTSl{cbP-i$)F>X90i!6DHqgQO7|9AiXQ zrRtFymBHtuRie5yoH*1aL^KPpLUpMrBCX05cR@m3ssLqoQC+G@B1QDSuS;vV>!;d+ z;hMb+{nxrYaOZe30lBhn%9j|roY2PGL(S306YA6)Mn5z*;#0<tx$jf%`;_}W<sN*> z%{SV6<s54f$Iqxr{dq$9jMOuK<mPT}zfrV%cgptt`|7J-ZzQ{RWF9YWcrtAL;@YaF z8hTFUkwyLAyn1$8T<a#EdFkwHHxrMyg{K!DzTB%~^P(6}R-czoSzF(pVzm@qZJoVj zM%Ubpp_6qf;ac(cmA#fvTRGGDPGE6IkS&~a&R+jYdzE}^{jdG)bzdh=9hUmy(8DKI zR2Q_27}Rof@cS2vD;sZB%;r8nP_`!Hfphp1#DlYrP24_gb;pfy-(MfdWi?E092-3U z*jrnEN}f9Ua4dV-Qo{`G{7zoD;n9n$$-h?DhId|BF?1()X++Jk4`MzXQ`K=j^hDpy zp85@SVQNt3lBs{jCEQNm_3fg@hlgFAm%DgF&2RPVHy3pcd~R;y<W<v?T=D){$`us3 z>AU?=TwKq+1pa9GP%NfMjVS)80_%NH6H&fHy$@+ndVdrbuMSrIkq)VRhwcE-b*T6w zHAzIB{yU`8i2TuGVXRlKn#L3af3ymVDbz&3AE}WDQtywnDiwcZKq7wx0L@k)f22nm z{E=3r;*X3-@JA{@Svc}XYD7_4t@lSdmFPQ!kU2feME=O268w=Gi6Hg<NT(9{Bcn?2 zN2;cXO7KSpq~0HCRVx0-pc45bJremNJ<{Nhv?>*UWKfCxkpT&HX+Y}zkxnJ@M@A&n yr3z3Mj{K3DMA^%#`*o@O52W2o#UB|?{GRHv+aH+~%cVW>Zhxfv`GG$|hw(2xc}ejA
literal 0 HcmV?d00001
diff --git a/dlls/mf/tests/mf_test.h b/dlls/mf/tests/mf_test.h index 0f23e2aeb02..05aa34e3d28 100644 --- a/dlls/mf/tests/mf_test.h +++ b/dlls/mf/tests/mf_test.h @@ -94,6 +94,7 @@ struct sample_desc const struct buffer_desc *buffers; DWORD repeat_count; BOOL todo_length; + BOOL todo_duration; LONGLONG todo_time; };
diff --git a/dlls/mf/tests/resource.rc b/dlls/mf/tests/resource.rc index e3df92e375f..fb0d0841f9b 100644 --- a/dlls/mf/tests/resource.rc +++ b/dlls/mf/tests/resource.rc @@ -63,6 +63,21 @@ mp3decdata.bin RCDATA mp3decdata.bin /* @makedep: h264data.bin */ h264data.bin RCDATA h264data.bin
+/* Generated with: + gst-launch-1.0 videotestsrc num-buffers=60 pattern=smpte100 ! \ + video/x-raw,format=I420,width=84,height=82,framerate=30000/1001 ! \ + videoflip method=clockwise ! videoconvert ! \ + x264enc ! filesink location=dlls/mf/tests/h264-concat-streams-001.bin + gst-launch-1.0 videotestsrc num-buffers=60 pattern=smpte100 ! \ + video/x-raw,format=I420,width=100,height=98,framerate=30000/1001 ! \ + videoflip method=clockwise ! videoconvert ! \ + x264enc ! filesink location=dlls/mf/tests/h264-concat-streams-002.bin + cat dlls/mf/tests/h264-concat-streams-001.bin dlls/mf/tests/h264-concat-streams-002.bin \ + >dlls/mf/tests/h264-concat-streams.bin +*/ +/* @makedep: h264-concat-streams.bin */ +h264-concat-streams.bin RCDATA h264-concat-streams.bin + /* Generated from running the tests on Windows */ /* @makedep: nv12frame.bmp */ nv12frame.bmp RCDATA nv12frame.bmp diff --git a/dlls/mf/tests/transform.c b/dlls/mf/tests/transform.c index b1ac3ffbb14..a47f9d50b62 100644 --- a/dlls/mf/tests/transform.c +++ b/dlls/mf/tests/transform.c @@ -1140,13 +1140,16 @@ static DWORD check_mf_media_buffer_(const char *file, int line, IMFMediaBuffer * todo_wine_if(expect->todo_length) ok_(file, line)(length == expect_length, "got length %#lx\n", length);
- if (*expect_data_len < length) - todo_wine_if(expect->todo_length) - ok_(file, line)(0, "missing %#lx bytes\n", length - *expect_data_len); - else if (!expect->compare) - diff = compare_bytes(data, &length, NULL, *expect_data); - else - diff = expect->compare(data, &length, &expect->rect, *expect_data); + if (*expect_data) + { + if (*expect_data_len < length) + todo_wine_if(expect->todo_length) + ok_(file, line)(0, "missing %#lx bytes\n", length - *expect_data_len); + else if (!expect->compare) + diff = compare_bytes(data, &length, NULL, *expect_data); + else + diff = expect->compare(data, &length, &expect->rect, *expect_data); + }
hr = IMFMediaBuffer_Unlock(buffer); ok_(file, line)(hr == S_OK, "Unlock returned %#lx\n", hr); @@ -1209,7 +1212,7 @@ static DWORD check_mf_sample_(const char *file, int line, IMFSample *sample, con timestamp = 0xdeadbeef; hr = IMFSample_GetSampleDuration(sample, ×tamp); ok_(file, line)(hr == S_OK, "GetSampleDuration returned %#lx\n", hr); - todo_wine_if(expect->todo_length) + todo_wine_if(expect->todo_duration && expect->todo_duration == timestamp) ok_(file, line)(llabs(timestamp - expect->sample_duration) <= 1, "got sample duration %I64d\n", timestamp);
@@ -1221,7 +1224,7 @@ static DWORD check_mf_sample_(const char *file, int line, IMFSample *sample, con todo_wine_if(expect->todo_length) ok_(file, line)(total_length == ctx.total_length, "got total length %#lx\n", total_length); - ok_(file, line)(*expect_data_len >= ctx.total_length, + ok_(file, line)(!*expect_data || *expect_data_len >= ctx.total_length, "missing %#lx data\n", ctx.total_length - *expect_data_len);
*expect_data = ctx.data; @@ -1243,10 +1246,10 @@ DWORD check_mf_sample_collection_(const char *file, int line, IMFCollection *sam DWORD count; HRESULT hr;
- load_resource(expect_data_filename, &ctx.data, &ctx.data_len); + if (expect_data_filename) load_resource(expect_data_filename, &ctx.data, &ctx.data_len); enum_mf_samples(samples, expect_samples, check_mf_sample_collection_enum, &ctx);
- dump_mf_sample_collection(samples, expect_samples, expect_data_filename); + if (expect_data_filename) dump_mf_sample_collection(samples, expect_samples, expect_data_filename);
hr = IMFCollection_GetElementCount(samples, &count); ok_(file, line)(hr == S_OK, "GetElementCount returned %#lx\n", hr); @@ -3159,7 +3162,7 @@ static void test_wma_decoder(void) }, { .attributes = output_sample_attributes + 1, /* not MFT_OUTPUT_DATA_BUFFER_INCOMPLETE */ - .sample_time = 2786394, .sample_duration = 464399, + .sample_time = 2786394, .sample_duration = 464399, .todo_duration = 928798, .buffer_count = 1, .buffers = output_buffer_desc + 1, .todo_length = TRUE, }, }; @@ -4016,6 +4019,149 @@ failed: CoUninitialize(); }
+static void test_h264_decoder_concat_streams(void) +{ + const struct buffer_desc output_buffer_desc[] = + { + {.length = 0x3600}, + {.length = 0x3600, .todo_length = TRUE}, + {.length = 0x4980}, + }; + const struct attribute_desc output_sample_attributes[] = + { + ATTR_UINT32(MFSampleExtension_CleanPoint, 1), + {0}, + }; + const struct sample_desc output_sample_desc[] = + { + { + .attributes = output_sample_attributes + 0, + .sample_time = 0, .sample_duration = 333667, + .buffer_count = 1, .buffers = output_buffer_desc + 0, .repeat_count = 57, + }, + { + .attributes = output_sample_attributes + 0, + .sample_time = 19352666, .sample_duration = 333667, + .buffer_count = 1, .buffers = output_buffer_desc + 1, .repeat_count = 1, + .todo_length = TRUE, + }, + { + .attributes = output_sample_attributes + 0, + .sample_time = 60 * 333667, .sample_duration = 333667, + .buffer_count = 1, .buffers = output_buffer_desc + 2, .repeat_count = 59, + }, + }; + + const BYTE *h264_encoded_data; + IMFCollection *output_samples; + ULONG h264_encoded_data_len; + DWORD output_status, count; + IMFAttributes *attributes; + IMFMediaType *media_type; + IMFTransform *transform; + IMFSample *input_sample; + HRESULT hr; + LONG ret; + + hr = CoInitialize(NULL); + ok(hr == S_OK, "Failed to initialize, hr %#lx.\n", hr); + + if (FAILED(hr = CoCreateInstance(&CLSID_MSH264DecoderMFT, NULL, CLSCTX_INPROC_SERVER, + &IID_IMFTransform, (void **)&transform))) + goto failed; + ok(hr == S_OK, "hr %#lx\n", hr); + + hr = IMFTransform_GetAttributes(transform, &attributes); + ok(hr == S_OK, "GetAttributes returned %#lx\n", hr); + hr = IMFAttributes_SetUINT32(attributes, &MF_LOW_LATENCY, 1); + ok(hr == S_OK, "SetUINT32 returned %#lx\n", hr); + IMFAttributes_Release(attributes); + + hr = IMFTransform_GetInputAvailableType(transform, 0, 0, &media_type); + ok(hr == S_OK, "got %#lx\n", hr); + hr = IMFTransform_SetInputType(transform, 0, media_type, 0); + ok(hr == S_OK, "got %#lx\n", hr); + IMFMediaType_Release(media_type); + + hr = IMFTransform_GetOutputAvailableType(transform, 0, 0, &media_type); + ok(hr == S_OK, "got %#lx\n", hr); + hr = IMFTransform_SetOutputType(transform, 0, media_type, 0); + ok(hr == S_OK, "got %#lx\n", hr); + IMFMediaType_Release(media_type); + + load_resource(L"h264-concat-streams.bin", &h264_encoded_data, &h264_encoded_data_len); + + hr = MFCreateCollection(&output_samples); + ok(hr == S_OK, "MFCreateCollection returned %#lx\n", hr); + + input_sample = next_h264_sample(&h264_encoded_data, &h264_encoded_data_len); + while (input_sample) + { + MFT_OUTPUT_STREAM_INFO info = {0}; + + hr = IMFTransform_ProcessInput(transform, 0, input_sample, 0); + ok(hr == S_OK || hr == MF_E_NOTACCEPTING, "ProcessInput returned %#lx\n", hr); + + if (hr == S_OK) + { + IMFSample_Release(input_sample); + + if (h264_encoded_data_len > 4) + input_sample = next_h264_sample(&h264_encoded_data, &h264_encoded_data_len); + else + { + hr = IMFTransform_ProcessMessage(transform, MFT_MESSAGE_COMMAND_DRAIN, 0); + ok(hr == S_OK, "ProcessMessage returned %#lx\n", hr); + input_sample = NULL; + } + } + + hr = IMFTransform_GetOutputStreamInfo(transform, 0, &info); + ok(hr == S_OK, "GetOutputStreamInfo returned %#lx\n", hr); + + while (hr != MF_E_TRANSFORM_NEED_MORE_INPUT) + { + MFT_OUTPUT_DATA_BUFFER data = {.pSample = create_sample(NULL, info.cbSize)}; + + hr = IMFTransform_ProcessOutput(transform, 0, 1, &data, &output_status); + todo_wine_if(hr == 0xd0000001) + ok(hr == S_OK || hr == MF_E_TRANSFORM_NEED_MORE_INPUT || hr == MF_E_TRANSFORM_STREAM_CHANGE, + "ProcessOutput returned %#lx\n", hr); + + if (hr == S_OK) + { + hr = IMFCollection_AddElement(output_samples, (IUnknown *)data.pSample); + ok(hr == S_OK, "AddElement returned %#lx\n", hr); + } + if (hr == MF_E_TRANSFORM_STREAM_CHANGE) + { + hr = IMFTransform_GetOutputAvailableType(transform, 0, 0, &media_type); + ok(hr == S_OK, "GetOutputAvailableType returned %#lx\n", hr); + hr = IMFTransform_SetOutputType(transform, 0, media_type, 0); + ok(hr == S_OK, "SetOutputType returned %#lx\n", hr); + hr = IMFTransform_GetOutputStreamInfo(transform, 0, &info); + ok(hr == S_OK, "GetOutputStreamInfo returned %#lx\n", hr); + } + + IMFSample_Release(data.pSample); + } + } + + hr = IMFCollection_GetElementCount(output_samples, &count); + ok(hr == S_OK, "GetElementCount returned %#lx\n", hr); + todo_wine + ok(count >= 0x77, "GetElementCount returned %#lx\n", count); + + ret = check_mf_sample_collection(output_samples, output_sample_desc, NULL); + ok(ret == 0, "got %lu%% diff\n", ret); + + ret = IMFTransform_Release(transform); + ok(ret == 0, "Release returned %lu\n", ret); + +failed: + CoUninitialize(); +} + static void test_audio_convert(void) { const GUID *const class_id = &CLSID_CResamplerMediaObject; @@ -4180,7 +4326,7 @@ static void test_audio_convert(void) }, { .attributes = output_sample_attributes + 1, /* not MFT_OUTPUT_DATA_BUFFER_INCOMPLETE */ - .sample_time = 9287980, .sample_duration = 897506, + .sample_time = 9287980, .sample_duration = 897506, .todo_duration = 897280, .buffer_count = 1, .buffers = output_buffer_desc + 1, .todo_length = TRUE, }, { @@ -7691,4 +7837,5 @@ START_TEST(transform) test_iv50_decoder();
test_h264_with_dxgi_manager(); + test_h264_decoder_concat_streams(); }
From: Rémi Bernon rbernon@codeweavers.com
Instead of constraining the output caps to the current resolution, which breaks when streams with different resolutions are concatenated. --- dlls/mf/tests/transform.c | 1 - dlls/winegstreamer/wg_transform.c | 6 +++++- 2 files changed, 5 insertions(+), 2 deletions(-)
diff --git a/dlls/mf/tests/transform.c b/dlls/mf/tests/transform.c index a47f9d50b62..9a50a07208e 100644 --- a/dlls/mf/tests/transform.c +++ b/dlls/mf/tests/transform.c @@ -4124,7 +4124,6 @@ static void test_h264_decoder_concat_streams(void) MFT_OUTPUT_DATA_BUFFER data = {.pSample = create_sample(NULL, info.cbSize)};
hr = IMFTransform_ProcessOutput(transform, 0, 1, &data, &output_status); - todo_wine_if(hr == 0xd0000001) ok(hr == S_OK || hr == MF_E_TRANSFORM_NEED_MORE_INPUT || hr == MF_E_TRANSFORM_STREAM_CHANGE, "ProcessOutput returned %#lx\n", hr);
diff --git a/dlls/winegstreamer/wg_transform.c b/dlls/winegstreamer/wg_transform.c index f75d1b4b6df..d68ef693092 100644 --- a/dlls/winegstreamer/wg_transform.c +++ b/dlls/winegstreamer/wg_transform.c @@ -56,6 +56,7 @@ struct wg_transform GstElement *video_flip;
guint output_plane_align; + struct wg_format output_format; struct wg_sample *output_wg_sample; GstAtomicQueue *output_queue; GstSample *output_sample; @@ -174,7 +175,8 @@ static gboolean transform_sink_query_cb(GstPad *pad, GstObject *parent, GstQuery gchar *str;
gst_query_parse_caps(query, &filter); - caps = gst_caps_ref(transform->output_caps); + if (!(caps = wg_format_to_caps(&transform->output_format))) + break;
if (filter) { @@ -300,6 +302,7 @@ NTSTATUS wg_transform_create(void *args) goto out; transform->input_max_length = 1; transform->output_plane_align = 0; + transform->output_format = output_format;
if (!(src_caps = wg_format_to_caps(&input_format))) goto out; @@ -496,6 +499,7 @@ NTSTATUS wg_transform_set_output_format(void *args) GST_ERROR("Failed to convert format %p to caps.", format); return STATUS_UNSUCCESSFUL; } + transform->output_format = *format;
if (gst_caps_is_always_compatible(transform->output_caps, caps)) {
From: Rémi Bernon rbernon@codeweavers.com
They are sometimes harmless, for instance when using openh264 decoder. --- dlls/winegstreamer/wg_transform.c | 21 +++++---------------- 1 file changed, 5 insertions(+), 16 deletions(-)
diff --git a/dlls/winegstreamer/wg_transform.c b/dlls/winegstreamer/wg_transform.c index d68ef693092..83626095f9e 100644 --- a/dlls/winegstreamer/wg_transform.c +++ b/dlls/winegstreamer/wg_transform.c @@ -744,30 +744,25 @@ static NTSTATUS read_transform_output_data(GstBuffer *buffer, GstCaps *caps, gsi
static bool get_transform_output(struct wg_transform *transform, struct wg_sample *sample) { - GstFlowReturn ret = GST_FLOW_OK; GstBuffer *input_buffer; + GstFlowReturn ret;
/* Provide the sample for transform_request_sample to pick it up */ InterlockedIncrement(&sample->refcount); InterlockedExchangePointer((void **)&transform->output_wg_sample, sample);
- while (!(transform->output_sample = gst_atomic_queue_pop(transform->output_queue))) + while (!(transform->output_sample = gst_atomic_queue_pop(transform->output_queue)) + && (input_buffer = gst_atomic_queue_pop(transform->input_queue))) { - if (!(input_buffer = gst_atomic_queue_pop(transform->input_queue))) - break; - if ((ret = gst_pad_push(transform->my_src, input_buffer))) - { - GST_ERROR("Failed to push transform input, error %d", ret); - break; - } + GST_WARNING("Failed to push transform input, error %d", ret); }
/* Remove the sample so transform_request_sample cannot use it */ if (InterlockedExchangePointer((void **)&transform->output_wg_sample, NULL)) InterlockedDecrement(&sample->refcount);
- return ret == GST_FLOW_OK; + return !!transform->output_sample; }
NTSTATUS wg_transform_read_data(void *args) @@ -782,12 +777,6 @@ NTSTATUS wg_transform_read_data(void *args) NTSTATUS status;
if (!transform->output_sample && !get_transform_output(transform, sample)) - { - wg_allocator_release_sample(transform->allocator, sample, false); - return STATUS_UNSUCCESSFUL; - } - - if (!transform->output_sample) { sample->size = 0; params->result = MF_E_TRANSFORM_NEED_MORE_INPUT;
From: Paul Gofman pgofman@codeweavers.com
--- dlls/winegstreamer/wg_transform.c | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-)
diff --git a/dlls/winegstreamer/wg_transform.c b/dlls/winegstreamer/wg_transform.c index 83626095f9e..b45c5ef9847 100644 --- a/dlls/winegstreamer/wg_transform.c +++ b/dlls/winegstreamer/wg_transform.c @@ -85,6 +85,11 @@ static GstFlowReturn transform_sink_chain_cb(GstPad *pad, GstObject *parent, Gst
GST_LOG("transform %p, buffer %p.", transform, buffer);
+ if (GST_BUFFER_PTS_IS_VALID(buffer)) + transform->segment.start = GST_BUFFER_PTS(buffer); + else if (GST_BUFFER_DURATION_IS_VALID(buffer)) + transform->segment.start += GST_BUFFER_DURATION(buffer); + if (!(sample = gst_sample_new(buffer, transform->output_caps, NULL, NULL))) { GST_ERROR("Failed to allocate transform %p output sample.", transform); @@ -304,6 +309,10 @@ NTSTATUS wg_transform_create(void *args) transform->output_plane_align = 0; transform->output_format = output_format;
+ /* We need to use GST_FORMAT_TIME here because it's the only format + * some elements such avdec_wmav2 correctly support. */ + gst_segment_init(&transform->segment, GST_FORMAT_TIME); + if (!(src_caps = wg_format_to_caps(&input_format))) goto out; if (!(template = gst_pad_template_new("src", GST_PAD_SRC, GST_PAD_ALWAYS, src_caps))) @@ -442,12 +451,6 @@ NTSTATUS wg_transform_create(void *args) if (!(event = gst_event_new_caps(src_caps)) || !gst_pad_push_event(transform->my_src, event)) goto out; - - /* We need to use GST_FORMAT_TIME here because it's the only format - * some elements such avdec_wmav2 correctly support. */ - gst_segment_init(&transform->segment, GST_FORMAT_TIME); - transform->segment.start = 0; - transform->segment.stop = -1; if (!(event = gst_event_new_segment(&transform->segment)) || !gst_pad_push_event(transform->my_src, event)) goto out;
From: Paul Gofman pgofman@codeweavers.com
--- dlls/mf/tests/transform.c | 8 ++++--- dlls/winegstreamer/gst_private.h | 1 + dlls/winegstreamer/h264_decoder.c | 13 +++++++---- dlls/winegstreamer/main.c | 15 +++++++++++++ dlls/winegstreamer/unix_private.h | 1 + dlls/winegstreamer/unixlib.h | 1 + dlls/winegstreamer/wg_parser.c | 1 + dlls/winegstreamer/wg_transform.c | 37 +++++++++++++++++++++++++++++++ 8 files changed, 70 insertions(+), 7 deletions(-)
diff --git a/dlls/mf/tests/transform.c b/dlls/mf/tests/transform.c index 9a50a07208e..a8e1cc83c62 100644 --- a/dlls/mf/tests/transform.c +++ b/dlls/mf/tests/transform.c @@ -3711,7 +3711,8 @@ static void test_h264_decoder(void) const struct sample_desc expect_output_sample_i420 = { .attributes = output_sample_attributes, - .sample_time = 333667, .sample_duration = 333667, .todo_time = 1334666 /* with VA-API */, + .sample_time = 333667, .todo_time = 1334666 /* with VA-API */, + .sample_duration = 333667, .todo_duration = 10000000 /* broken on the testbot libav */, .buffer_count = 1, .buffers = &output_buffer_desc_i420, };
@@ -3879,10 +3880,10 @@ static void test_h264_decoder(void)
hr = IMFTransform_ProcessMessage(transform, MFT_MESSAGE_COMMAND_DRAIN, 0); ok(hr == S_OK, "ProcessMessage returned %#lx\n", hr); + hr = IMFTransform_ProcessMessage(transform, MFT_MESSAGE_COMMAND_DRAIN, 0); + ok(hr == S_OK, "ProcessMessage returned %#lx\n", hr); } - todo_wine ok(i == 2, "got %lu iterations\n", i); - todo_wine ok(h264_encoded_data_len == 2425, "got h264_encoded_data_len %lu\n", h264_encoded_data_len); ok(hr == MF_E_TRANSFORM_STREAM_CHANGE, "ProcessOutput returned %#lx\n", hr); ok(output_status == MFT_OUTPUT_DATA_BUFFER_FORMAT_CHANGE, "got output[0].dwStatus %#lx\n", output_status); @@ -3981,6 +3982,7 @@ static void test_h264_decoder(void) ok(ref == 1, "Release returned %ld\n", ref);
ret = check_mf_sample_collection(output_samples, &expect_output_sample_i420, L"i420frame.bmp"); + todo_wine /* wg_transform_set_output_format() should convert already processed samples instead of dropping */ ok(ret == 0, "got %lu%% diff\n", ret); IMFCollection_Release(output_samples);
diff --git a/dlls/winegstreamer/gst_private.h b/dlls/winegstreamer/gst_private.h index 54f59aa708a..9a559f58507 100644 --- a/dlls/winegstreamer/gst_private.h +++ b/dlls/winegstreamer/gst_private.h @@ -106,6 +106,7 @@ struct wg_transform *wg_transform_create(const struct wg_format *input_format, void wg_transform_destroy(struct wg_transform *transform); bool wg_transform_set_output_format(struct wg_transform *transform, struct wg_format *format); bool wg_transform_get_status(struct wg_transform *transform, bool *accepts_input); +HRESULT wg_transform_drain(struct wg_transform *transform);
unsigned int wg_format_get_max_size(const struct wg_format *format);
diff --git a/dlls/winegstreamer/h264_decoder.c b/dlls/winegstreamer/h264_decoder.c index 1e18aff2635..70602088bfd 100644 --- a/dlls/winegstreamer/h264_decoder.c +++ b/dlls/winegstreamer/h264_decoder.c @@ -618,8 +618,9 @@ static HRESULT WINAPI transform_ProcessMessage(IMFTransform *iface, MFT_MESSAGE_
TRACE("iface %p, message %#x, param %Ix.\n", iface, message, param);
- if (message == MFT_MESSAGE_SET_D3D_MANAGER) + switch (message) { + case MFT_MESSAGE_SET_D3D_MANAGER: if (FAILED(hr = IMFVideoSampleAllocatorEx_SetDirectXManager(decoder->allocator, (IUnknown *)param))) return hr;
@@ -629,10 +630,14 @@ static HRESULT WINAPI transform_ProcessMessage(IMFTransform *iface, MFT_MESSAGE_ else decoder->output_info.dwFlags &= ~MFT_OUTPUT_STREAM_PROVIDES_SAMPLES; return S_OK; - }
- FIXME("Ignoring message %#x.\n", message); - return S_OK; + case MFT_MESSAGE_COMMAND_DRAIN: + return wg_transform_drain(decoder->wg_transform); + + default: + FIXME("Ignoring message %#x.\n", message); + return S_OK; + } }
static HRESULT WINAPI transform_ProcessInput(IMFTransform *iface, DWORD id, IMFSample *sample, DWORD flags) diff --git a/dlls/winegstreamer/main.c b/dlls/winegstreamer/main.c index 2e7763872d0..955644b6a60 100644 --- a/dlls/winegstreamer/main.c +++ b/dlls/winegstreamer/main.c @@ -426,6 +426,21 @@ bool wg_transform_set_output_format(struct wg_transform *transform, struct wg_fo return !WINE_UNIX_CALL(unix_wg_transform_set_output_format, ¶ms); }
+HRESULT wg_transform_drain(struct wg_transform *transform) +{ + NTSTATUS status; + + TRACE("transform %p.\n", transform); + + if ((status = WINE_UNIX_CALL(unix_wg_transform_drain, transform))) + { + WARN("wg_transform_drain returned status %#lx\n", status); + return HRESULT_FROM_NT(status); + } + + return S_OK; +} + #define ALIGN(n, alignment) (((n) + (alignment) - 1) & ~((alignment) - 1))
unsigned int wg_format_get_stride(const struct wg_format *format) diff --git a/dlls/winegstreamer/unix_private.h b/dlls/winegstreamer/unix_private.h index 962668045c1..dcc89298748 100644 --- a/dlls/winegstreamer/unix_private.h +++ b/dlls/winegstreamer/unix_private.h @@ -53,6 +53,7 @@ extern NTSTATUS wg_transform_set_output_format(void *args) DECLSPEC_HIDDEN; extern NTSTATUS wg_transform_push_data(void *args) DECLSPEC_HIDDEN; extern NTSTATUS wg_transform_read_data(void *args) DECLSPEC_HIDDEN; extern NTSTATUS wg_transform_get_status(void *args) DECLSPEC_HIDDEN; +extern NTSTATUS wg_transform_drain(void *args) DECLSPEC_HIDDEN;
/* wg_allocator.c */
diff --git a/dlls/winegstreamer/unixlib.h b/dlls/winegstreamer/unixlib.h index e20fdd7256b..c712be0ed13 100644 --- a/dlls/winegstreamer/unixlib.h +++ b/dlls/winegstreamer/unixlib.h @@ -376,6 +376,7 @@ enum unix_funcs unix_wg_transform_push_data, unix_wg_transform_read_data, unix_wg_transform_get_status, + unix_wg_transform_drain, };
#endif /* __WINE_WINEGSTREAMER_UNIXLIB_H */ diff --git a/dlls/winegstreamer/wg_parser.c b/dlls/winegstreamer/wg_parser.c index ef274b1dc27..c55685931ae 100644 --- a/dlls/winegstreamer/wg_parser.c +++ b/dlls/winegstreamer/wg_parser.c @@ -1939,4 +1939,5 @@ const unixlib_entry_t __wine_unix_call_funcs[] = X(wg_transform_push_data), X(wg_transform_read_data), X(wg_transform_get_status), + X(wg_transform_drain), }; diff --git a/dlls/winegstreamer/wg_transform.c b/dlls/winegstreamer/wg_transform.c index b45c5ef9847..3361f94ead7 100644 --- a/dlls/winegstreamer/wg_transform.c +++ b/dlls/winegstreamer/wg_transform.c @@ -871,3 +871,40 @@ NTSTATUS wg_transform_get_status(void *args) params->accepts_input = gst_atomic_queue_length(transform->input_queue) < transform->input_max_length; return STATUS_SUCCESS; } + +NTSTATUS wg_transform_drain(void *args) +{ + struct wg_transform *transform = args; + GstBuffer *input_buffer; + GstFlowReturn ret; + GstEvent *event; + + GST_LOG("transform %p", transform); + + while ((input_buffer = gst_atomic_queue_pop(transform->input_queue))) + { + if ((ret = gst_pad_push(transform->my_src, input_buffer))) + GST_WARNING("Failed to push transform input, error %d", ret); + } + + if (!gst_pad_peer_query(transform->my_src, transform->drain_query)) + goto error; + if (!(event = gst_event_new_segment_done(GST_FORMAT_TIME, transform->segment.start)) + || !gst_pad_push_event(transform->my_src, event)) + goto error; + if (!(event = gst_event_new_eos()) + || !gst_pad_push_event(transform->my_src, event)) + goto error; + if (!(event = gst_event_new_stream_start("stream")) + || !gst_pad_push_event(transform->my_src, event)) + goto error; + if (!(event = gst_event_new_segment(&transform->segment)) + || !gst_pad_push_event(transform->my_src, event)) + goto error; + + return STATUS_SUCCESS; + +error: + GST_ERROR("Failed to drain transform %p.", transform); + return STATUS_UNSUCCESSFUL; +}
From: Paul Gofman pgofman@codeweavers.com
--- dlls/winegstreamer/gst_private.h | 1 + dlls/winegstreamer/h264_decoder.c | 3 +++ dlls/winegstreamer/main.c | 15 +++++++++++++++ dlls/winegstreamer/unix_private.h | 1 + dlls/winegstreamer/unixlib.h | 1 + dlls/winegstreamer/wg_parser.c | 1 + dlls/winegstreamer/wg_transform.c | 24 ++++++++++++++++++++++++ 7 files changed, 46 insertions(+)
diff --git a/dlls/winegstreamer/gst_private.h b/dlls/winegstreamer/gst_private.h index 9a559f58507..af4590af8bb 100644 --- a/dlls/winegstreamer/gst_private.h +++ b/dlls/winegstreamer/gst_private.h @@ -107,6 +107,7 @@ void wg_transform_destroy(struct wg_transform *transform); bool wg_transform_set_output_format(struct wg_transform *transform, struct wg_format *format); bool wg_transform_get_status(struct wg_transform *transform, bool *accepts_input); HRESULT wg_transform_drain(struct wg_transform *transform); +HRESULT wg_transform_flush(struct wg_transform *transform);
unsigned int wg_format_get_max_size(const struct wg_format *format);
diff --git a/dlls/winegstreamer/h264_decoder.c b/dlls/winegstreamer/h264_decoder.c index 70602088bfd..9c68105279a 100644 --- a/dlls/winegstreamer/h264_decoder.c +++ b/dlls/winegstreamer/h264_decoder.c @@ -634,6 +634,9 @@ static HRESULT WINAPI transform_ProcessMessage(IMFTransform *iface, MFT_MESSAGE_ case MFT_MESSAGE_COMMAND_DRAIN: return wg_transform_drain(decoder->wg_transform);
+ case MFT_MESSAGE_COMMAND_FLUSH: + return wg_transform_flush(decoder->wg_transform); + default: FIXME("Ignoring message %#x.\n", message); return S_OK; diff --git a/dlls/winegstreamer/main.c b/dlls/winegstreamer/main.c index 955644b6a60..bd4f7108387 100644 --- a/dlls/winegstreamer/main.c +++ b/dlls/winegstreamer/main.c @@ -441,6 +441,21 @@ HRESULT wg_transform_drain(struct wg_transform *transform) return S_OK; }
+HRESULT wg_transform_flush(struct wg_transform *transform) +{ + NTSTATUS status; + + TRACE("transform %p.\n", transform); + + if ((status = WINE_UNIX_CALL(unix_wg_transform_flush, transform))) + { + WARN("wg_transform_flush returned status %#lx\n", status); + return HRESULT_FROM_NT(status); + } + + return S_OK; +} + #define ALIGN(n, alignment) (((n) + (alignment) - 1) & ~((alignment) - 1))
unsigned int wg_format_get_stride(const struct wg_format *format) diff --git a/dlls/winegstreamer/unix_private.h b/dlls/winegstreamer/unix_private.h index dcc89298748..808b59dba57 100644 --- a/dlls/winegstreamer/unix_private.h +++ b/dlls/winegstreamer/unix_private.h @@ -54,6 +54,7 @@ extern NTSTATUS wg_transform_push_data(void *args) DECLSPEC_HIDDEN; extern NTSTATUS wg_transform_read_data(void *args) DECLSPEC_HIDDEN; extern NTSTATUS wg_transform_get_status(void *args) DECLSPEC_HIDDEN; extern NTSTATUS wg_transform_drain(void *args) DECLSPEC_HIDDEN; +extern NTSTATUS wg_transform_flush(void *args) DECLSPEC_HIDDEN;
/* wg_allocator.c */
diff --git a/dlls/winegstreamer/unixlib.h b/dlls/winegstreamer/unixlib.h index c712be0ed13..e03085cd4f7 100644 --- a/dlls/winegstreamer/unixlib.h +++ b/dlls/winegstreamer/unixlib.h @@ -377,6 +377,7 @@ enum unix_funcs unix_wg_transform_read_data, unix_wg_transform_get_status, unix_wg_transform_drain, + unix_wg_transform_flush, };
#endif /* __WINE_WINEGSTREAMER_UNIXLIB_H */ diff --git a/dlls/winegstreamer/wg_parser.c b/dlls/winegstreamer/wg_parser.c index c55685931ae..f043c4e18b4 100644 --- a/dlls/winegstreamer/wg_parser.c +++ b/dlls/winegstreamer/wg_parser.c @@ -1940,4 +1940,5 @@ const unixlib_entry_t __wine_unix_call_funcs[] = X(wg_transform_read_data), X(wg_transform_get_status), X(wg_transform_drain), + X(wg_transform_flush), }; diff --git a/dlls/winegstreamer/wg_transform.c b/dlls/winegstreamer/wg_transform.c index 3361f94ead7..e9e6350dd9f 100644 --- a/dlls/winegstreamer/wg_transform.c +++ b/dlls/winegstreamer/wg_transform.c @@ -908,3 +908,27 @@ error: GST_ERROR("Failed to drain transform %p.", transform); return STATUS_UNSUCCESSFUL; } + +NTSTATUS wg_transform_flush(void *args) +{ + struct wg_transform *transform = args; + GstBuffer *input_buffer; + GstSample *sample; + NTSTATUS status; + + GST_LOG("transform %p", transform); + + while ((input_buffer = gst_atomic_queue_pop(transform->input_queue))) + gst_buffer_unref(input_buffer); + + if ((status = wg_transform_drain(transform))) + return status; + + while ((sample = gst_atomic_queue_pop(transform->output_queue))) + gst_sample_unref(sample); + if ((sample = transform->output_sample)) + gst_sample_unref(sample); + transform->output_sample = NULL; + + return STATUS_SUCCESS; +}
From: Rémi Bernon rbernon@codeweavers.com
--- dlls/winegstreamer/aac_decoder.c | 6 ++++-- dlls/winegstreamer/color_convert.c | 6 ++++-- dlls/winegstreamer/gst_private.h | 2 +- dlls/winegstreamer/h264_decoder.c | 9 +++++++-- dlls/winegstreamer/main.c | 3 ++- dlls/winegstreamer/quartz_transform.c | 9 ++++++--- dlls/winegstreamer/resampler.c | 6 ++++-- dlls/winegstreamer/unixlib.h | 6 ++++++ dlls/winegstreamer/video_decoder.c | 3 ++- dlls/winegstreamer/video_processor.c | 6 ++++-- dlls/winegstreamer/wg_transform.c | 12 ++++++------ dlls/winegstreamer/wma_decoder.c | 6 ++++-- dlls/winegstreamer/wmv_decoder.c | 6 ++++-- 13 files changed, 54 insertions(+), 26 deletions(-)
diff --git a/dlls/winegstreamer/aac_decoder.c b/dlls/winegstreamer/aac_decoder.c index d79ede69c9d..1fc545fc283 100644 --- a/dlls/winegstreamer/aac_decoder.c +++ b/dlls/winegstreamer/aac_decoder.c @@ -67,6 +67,7 @@ static struct aac_decoder *impl_from_IMFTransform(IMFTransform *iface) static HRESULT try_create_wg_transform(struct aac_decoder *decoder) { struct wg_format input_format, output_format; + struct wg_transform_attrs attrs = {0};
if (decoder->wg_transform) wg_transform_destroy(decoder->wg_transform); @@ -80,7 +81,7 @@ static HRESULT try_create_wg_transform(struct aac_decoder *decoder) if (output_format.major_type == WG_MAJOR_TYPE_UNKNOWN) return MF_E_INVALIDMEDIATYPE;
- if (!(decoder->wg_transform = wg_transform_create(&input_format, &output_format))) + if (!(decoder->wg_transform = wg_transform_create(&input_format, &output_format, &attrs))) return E_FAIL;
return S_OK; @@ -625,13 +626,14 @@ HRESULT aac_decoder_create(REFIID riid, void **ret) }, }; static const struct wg_format input_format = {.major_type = WG_MAJOR_TYPE_AUDIO_MPEG4}; + struct wg_transform_attrs attrs = {0}; struct wg_transform *transform; struct aac_decoder *decoder; HRESULT hr;
TRACE("riid %s, ret %p.\n", debugstr_guid(riid), ret);
- if (!(transform = wg_transform_create(&input_format, &output_format))) + if (!(transform = wg_transform_create(&input_format, &output_format, &attrs))) { ERR_(winediag)("GStreamer doesn't support WMA decoding, please install appropriate plugins\n"); return E_FAIL; diff --git a/dlls/winegstreamer/color_convert.c b/dlls/winegstreamer/color_convert.c index ed591c8e669..624191daf33 100644 --- a/dlls/winegstreamer/color_convert.c +++ b/dlls/winegstreamer/color_convert.c @@ -98,6 +98,7 @@ static inline struct color_convert *impl_from_IUnknown(IUnknown *iface) static HRESULT try_create_wg_transform(struct color_convert *impl) { struct wg_format input_format, output_format; + struct wg_transform_attrs attrs = {0};
if (impl->wg_transform) wg_transform_destroy(impl->wg_transform); @@ -111,7 +112,7 @@ static HRESULT try_create_wg_transform(struct color_convert *impl) if (output_format.major_type == WG_MAJOR_TYPE_UNKNOWN) return MF_E_INVALIDMEDIATYPE;
- if (!(impl->wg_transform = wg_transform_create(&input_format, &output_format))) + if (!(impl->wg_transform = wg_transform_create(&input_format, &output_format, &attrs))) return E_FAIL;
return S_OK; @@ -936,13 +937,14 @@ HRESULT color_convert_create(IUnknown *outer, IUnknown **out) .height = 1080, }, }; + struct wg_transform_attrs attrs = {0}; struct wg_transform *transform; struct color_convert *impl; HRESULT hr;
TRACE("outer %p, out %p.\n", outer, out);
- if (!(transform = wg_transform_create(&input_format, &output_format))) + if (!(transform = wg_transform_create(&input_format, &output_format, &attrs))) { ERR_(winediag)("GStreamer doesn't support video conversion, please install appropriate plugins.\n"); return E_FAIL; diff --git a/dlls/winegstreamer/gst_private.h b/dlls/winegstreamer/gst_private.h index af4590af8bb..d397ab21ca0 100644 --- a/dlls/winegstreamer/gst_private.h +++ b/dlls/winegstreamer/gst_private.h @@ -102,7 +102,7 @@ void wg_parser_stream_seek(struct wg_parser_stream *stream, double rate, uint64_t start_pos, uint64_t stop_pos, DWORD start_flags, DWORD stop_flags);
struct wg_transform *wg_transform_create(const struct wg_format *input_format, - const struct wg_format *output_format); + const struct wg_format *output_format, const struct wg_transform_attrs *attrs); void wg_transform_destroy(struct wg_transform *transform); bool wg_transform_set_output_format(struct wg_transform *transform, struct wg_format *format); bool wg_transform_get_status(struct wg_transform *transform, bool *accepts_input); diff --git a/dlls/winegstreamer/h264_decoder.c b/dlls/winegstreamer/h264_decoder.c index 9c68105279a..041421a427d 100644 --- a/dlls/winegstreamer/h264_decoder.c +++ b/dlls/winegstreamer/h264_decoder.c @@ -75,6 +75,10 @@ static struct h264_decoder *impl_from_IMFTransform(IMFTransform *iface)
static HRESULT try_create_wg_transform(struct h264_decoder *decoder) { + struct wg_transform_attrs attrs = + { + .output_plane_align = 15, + }; struct wg_format input_format; struct wg_format output_format;
@@ -98,7 +102,7 @@ static HRESULT try_create_wg_transform(struct h264_decoder *decoder) output_format.u.video.fps_d = 0; output_format.u.video.fps_n = 0;
- if (!(decoder->wg_transform = wg_transform_create(&input_format, &output_format))) + if (!(decoder->wg_transform = wg_transform_create(&input_format, &output_format, &attrs))) return E_FAIL;
return S_OK; @@ -812,13 +816,14 @@ HRESULT h264_decoder_create(REFIID riid, void **ret) }, }; static const struct wg_format input_format = {.major_type = WG_MAJOR_TYPE_VIDEO_H264}; + struct wg_transform_attrs attrs = {0}; struct wg_transform *transform; struct h264_decoder *decoder; HRESULT hr;
TRACE("riid %s, ret %p.\n", debugstr_guid(riid), ret);
- if (!(transform = wg_transform_create(&input_format, &output_format))) + if (!(transform = wg_transform_create(&input_format, &output_format, &attrs))) { ERR_(winediag)("GStreamer doesn't support H.264 decoding, please install appropriate plugins\n"); return E_FAIL; diff --git a/dlls/winegstreamer/main.c b/dlls/winegstreamer/main.c index bd4f7108387..88179feeb56 100644 --- a/dlls/winegstreamer/main.c +++ b/dlls/winegstreamer/main.c @@ -337,12 +337,13 @@ void wg_parser_stream_seek(struct wg_parser_stream *stream, double rate, }
struct wg_transform *wg_transform_create(const struct wg_format *input_format, - const struct wg_format *output_format) + const struct wg_format *output_format, const struct wg_transform_attrs *attrs) { struct wg_transform_create_params params = { .input_format = input_format, .output_format = output_format, + .attrs = attrs, };
TRACE("input_format %p, output_format %p.\n", input_format, output_format); diff --git a/dlls/winegstreamer/quartz_transform.c b/dlls/winegstreamer/quartz_transform.c index 09ad4862410..84f6bbd6361 100644 --- a/dlls/winegstreamer/quartz_transform.c +++ b/dlls/winegstreamer/quartz_transform.c @@ -98,6 +98,7 @@ static HRESULT transform_init_stream(struct strmbase_filter *iface) { struct transform *filter = impl_from_strmbase_filter(iface); struct wg_format input_format, output_format; + struct wg_transform_attrs attrs = {0}; HRESULT hr;
if (filter->source.pin.peer) @@ -111,7 +112,7 @@ static HRESULT transform_init_stream(struct strmbase_filter *iface) if (FAILED(hr = wg_sample_queue_create(&filter->sample_queue))) return hr;
- filter->transform = wg_transform_create(&input_format, &output_format); + filter->transform = wg_transform_create(&input_format, &output_format, &attrs); if (!filter->transform) { wg_sample_queue_destroy(filter->sample_queue); @@ -710,11 +711,12 @@ HRESULT mpeg_audio_codec_create(IUnknown *outer, IUnknown **out) .rate = 44100, }, }; + struct wg_transform_attrs attrs = {0}; struct wg_transform *transform; struct transform *object; HRESULT hr;
- transform = wg_transform_create(&input_format, &output_format); + transform = wg_transform_create(&input_format, &output_format, &attrs); if (!transform) { ERR_(winediag)("GStreamer doesn't support MPEG-1 audio decoding, please install appropriate plugins.\n"); @@ -844,11 +846,12 @@ HRESULT mpeg_layer3_decoder_create(IUnknown *outer, IUnknown **out) .rate = 44100, }, }; + struct wg_transform_attrs attrs = {0}; struct wg_transform *transform; struct transform *object; HRESULT hr;
- transform = wg_transform_create(&input_format, &output_format); + transform = wg_transform_create(&input_format, &output_format, &attrs); if (!transform) { ERR_(winediag)("GStreamer doesn't support MPEG-1 audio decoding, please install appropriate plugins.\n"); diff --git a/dlls/winegstreamer/resampler.c b/dlls/winegstreamer/resampler.c index 88e9727ff21..7ed8cf48fbf 100644 --- a/dlls/winegstreamer/resampler.c +++ b/dlls/winegstreamer/resampler.c @@ -56,6 +56,7 @@ struct resampler static HRESULT try_create_wg_transform(struct resampler *impl) { struct wg_format input_format, output_format; + struct wg_transform_attrs attrs = {0};
if (impl->wg_transform) wg_transform_destroy(impl->wg_transform); @@ -69,7 +70,7 @@ static HRESULT try_create_wg_transform(struct resampler *impl) if (output_format.major_type == WG_MAJOR_TYPE_UNKNOWN) return MF_E_INVALIDMEDIATYPE;
- if (!(impl->wg_transform = wg_transform_create(&input_format, &output_format))) + if (!(impl->wg_transform = wg_transform_create(&input_format, &output_format, &attrs))) return E_FAIL;
return S_OK; @@ -895,13 +896,14 @@ HRESULT resampler_create(IUnknown *outer, IUnknown **out) .rate = 44100, }, }; + struct wg_transform_attrs attrs = {0}; struct wg_transform *transform; struct resampler *impl; HRESULT hr;
TRACE("outer %p, out %p.\n", outer, out);
- if (!(transform = wg_transform_create(&input_format, &output_format))) + if (!(transform = wg_transform_create(&input_format, &output_format, &attrs))) { ERR_(winediag)("GStreamer doesn't support audio resampling, please install appropriate plugins.\n"); return E_FAIL; diff --git a/dlls/winegstreamer/unixlib.h b/dlls/winegstreamer/unixlib.h index e03085cd4f7..15b5dcff352 100644 --- a/dlls/winegstreamer/unixlib.h +++ b/dlls/winegstreamer/unixlib.h @@ -305,11 +305,17 @@ struct wg_parser_stream_seek_params DWORD start_flags, stop_flags; };
+struct wg_transform_attrs +{ + UINT32 output_plane_align; +}; + struct wg_transform_create_params { struct wg_transform *transform; const struct wg_format *input_format; const struct wg_format *output_format; + const struct wg_transform_attrs *attrs; };
struct wg_transform_push_data_params diff --git a/dlls/winegstreamer/video_decoder.c b/dlls/winegstreamer/video_decoder.c index 04d175c7910..abcadf90a32 100644 --- a/dlls/winegstreamer/video_decoder.c +++ b/dlls/winegstreamer/video_decoder.c @@ -68,6 +68,7 @@ static struct video_decoder *impl_from_IMFTransform(IMFTransform *iface)
static HRESULT try_create_wg_transform(struct video_decoder *decoder) { + struct wg_transform_attrs attrs = {0}; struct wg_format input_format; struct wg_format output_format;
@@ -86,7 +87,7 @@ static HRESULT try_create_wg_transform(struct video_decoder *decoder) output_format.u.video.fps_d = 0; output_format.u.video.fps_n = 0;
- if (!(decoder->wg_transform = wg_transform_create(&input_format, &output_format))) + if (!(decoder->wg_transform = wg_transform_create(&input_format, &output_format, &attrs))) { ERR("Failed to create transform with input major_type %u.\n", input_format.major_type); return E_FAIL; diff --git a/dlls/winegstreamer/video_processor.c b/dlls/winegstreamer/video_processor.c index 21bc46b0bb3..3cb0ccc090b 100644 --- a/dlls/winegstreamer/video_processor.c +++ b/dlls/winegstreamer/video_processor.c @@ -88,6 +88,7 @@ struct video_processor static HRESULT try_create_wg_transform(struct video_processor *impl) { struct wg_format input_format, output_format; + struct wg_transform_attrs attrs = {0};
if (impl->wg_transform) wg_transform_destroy(impl->wg_transform); @@ -101,7 +102,7 @@ static HRESULT try_create_wg_transform(struct video_processor *impl) if (output_format.major_type == WG_MAJOR_TYPE_UNKNOWN) return MF_E_INVALIDMEDIATYPE;
- if (!(impl->wg_transform = wg_transform_create(&input_format, &output_format))) + if (!(impl->wg_transform = wg_transform_create(&input_format, &output_format, &attrs))) return E_FAIL;
return S_OK; @@ -602,13 +603,14 @@ HRESULT video_processor_create(REFIID riid, void **ret) .height = 1080, }, }; + struct wg_transform_attrs attrs = {0}; struct wg_transform *transform; struct video_processor *impl; HRESULT hr;
TRACE("riid %s, ret %p.\n", debugstr_guid(riid), ret);
- if (!(transform = wg_transform_create(&input_format, &output_format))) + if (!(transform = wg_transform_create(&input_format, &output_format, &attrs))) { ERR_(winediag)("GStreamer doesn't support video conversion, please install appropriate plugins.\n"); return E_FAIL; diff --git a/dlls/winegstreamer/wg_transform.c b/dlls/winegstreamer/wg_transform.c index e9e6350dd9f..64310747d82 100644 --- a/dlls/winegstreamer/wg_transform.c +++ b/dlls/winegstreamer/wg_transform.c @@ -43,6 +43,8 @@
struct wg_transform { + struct wg_transform_attrs attrs; + GstElement *container; GstAllocator *allocator; GstPad *my_src, *my_sink; @@ -55,7 +57,6 @@ struct wg_transform bool input_is_flipped; GstElement *video_flip;
- guint output_plane_align; struct wg_format output_format; struct wg_sample *output_wg_sample; GstAtomicQueue *output_queue; @@ -116,7 +117,7 @@ static gboolean transform_sink_query_cb(GstPad *pad, GstObject *parent, GstQuery { case GST_QUERY_ALLOCATION: { - gsize plane_align = transform->output_plane_align; + gsize plane_align = transform->attrs.output_plane_align; GstStructure *config, *params; GstVideoAlignment align; gboolean needs_pool; @@ -305,8 +306,8 @@ NTSTATUS wg_transform_create(void *args) goto out; if (!(transform->allocator = wg_allocator_create(transform_request_sample, transform))) goto out; + transform->attrs = *params->attrs; transform->input_max_length = 1; - transform->output_plane_align = 0; transform->output_format = output_format;
/* We need to use GST_FORMAT_TIME here because it's the only format @@ -353,7 +354,6 @@ NTSTATUS wg_transform_create(void *args) * to match its expectations. */ transform->input_max_length = 16; - transform->output_plane_align = 15; if (!(element = create_element("h264parse", "base")) || !append_element(transform->container, element, &first, &last)) goto out; @@ -797,7 +797,7 @@ NTSTATUS wg_transform_read_data(void *args)
if (format) { - gsize plane_align = transform->output_plane_align; + gsize plane_align = transform->attrs.output_plane_align; GstVideoAlignment align; GstVideoInfo info;
@@ -829,7 +829,7 @@ NTSTATUS wg_transform_read_data(void *args) }
if ((status = read_transform_output_data(output_buffer, output_caps, - transform->output_plane_align, sample))) + transform->attrs.output_plane_align, sample))) { wg_allocator_release_sample(transform->allocator, sample, false); return status; diff --git a/dlls/winegstreamer/wma_decoder.c b/dlls/winegstreamer/wma_decoder.c index 4a0d5f2592e..dd06faf494d 100644 --- a/dlls/winegstreamer/wma_decoder.c +++ b/dlls/winegstreamer/wma_decoder.c @@ -70,6 +70,7 @@ static inline struct wma_decoder *impl_from_IUnknown(IUnknown *iface) static HRESULT try_create_wg_transform(struct wma_decoder *decoder) { struct wg_format input_format, output_format; + struct wg_transform_attrs attrs = {0};
if (decoder->wg_transform) wg_transform_destroy(decoder->wg_transform); @@ -83,7 +84,7 @@ static HRESULT try_create_wg_transform(struct wma_decoder *decoder) if (output_format.major_type == WG_MAJOR_TYPE_UNKNOWN) return MF_E_INVALIDMEDIATYPE;
- if (!(decoder->wg_transform = wg_transform_create(&input_format, &output_format))) + if (!(decoder->wg_transform = wg_transform_create(&input_format, &output_format, &attrs))) return E_FAIL;
return S_OK; @@ -851,13 +852,14 @@ HRESULT wma_decoder_create(IUnknown *outer, IUnknown **out) }, }; static const struct wg_format input_format = {.major_type = WG_MAJOR_TYPE_AUDIO_WMA}; + struct wg_transform_attrs attrs = {0}; struct wg_transform *transform; struct wma_decoder *decoder; HRESULT hr;
TRACE("outer %p, out %p.\n", outer, out);
- if (!(transform = wg_transform_create(&input_format, &output_format))) + if (!(transform = wg_transform_create(&input_format, &output_format, &attrs))) { ERR_(winediag)("GStreamer doesn't support WMA decoding, please install appropriate plugins\n"); return E_FAIL; diff --git a/dlls/winegstreamer/wmv_decoder.c b/dlls/winegstreamer/wmv_decoder.c index a22de7ccb7f..32dbbbbe686 100644 --- a/dlls/winegstreamer/wmv_decoder.c +++ b/dlls/winegstreamer/wmv_decoder.c @@ -540,6 +540,7 @@ static HRESULT WINAPI media_object_SetOutputType(IMediaObject *iface, DWORD inde const DMO_MEDIA_TYPE *type, DWORD flags) { struct wmv_decoder *decoder = impl_from_IMediaObject(iface); + struct wg_transform_attrs attrs = {0}; struct wg_format wg_format; unsigned int i;
@@ -593,7 +594,7 @@ static HRESULT WINAPI media_object_SetOutputType(IMediaObject *iface, DWORD inde wg_transform_destroy(decoder->wg_transform); decoder->wg_transform = NULL; } - if (!(decoder->wg_transform = wg_transform_create(&decoder->input_format, &decoder->output_format))) + if (!(decoder->wg_transform = wg_transform_create(&decoder->input_format, &decoder->output_format, &attrs))) return E_FAIL;
return S_OK; @@ -885,13 +886,14 @@ HRESULT wmv_decoder_create(IUnknown *outer, IUnknown **out) .height = 1080, }, }; + struct wg_transform_attrs attrs = {0}; struct wg_transform *transform; struct wmv_decoder *decoder; HRESULT hr;
TRACE("outer %p, out %p.\n", outer, out);
- if (!(transform = wg_transform_create(&input_format, &output_format))) + if (!(transform = wg_transform_create(&input_format, &output_format, &attrs))) { ERR_(winediag)("GStreamer doesn't support WMV decoding, please install appropriate plugins.\n"); return E_FAIL;
From: Rémi Bernon rbernon@codeweavers.com
--- dlls/winegstreamer/h264_decoder.c | 6 ++++++ dlls/winegstreamer/unixlib.h | 1 + dlls/winegstreamer/wg_transform.c | 12 ++---------- 3 files changed, 9 insertions(+), 10 deletions(-)
diff --git a/dlls/winegstreamer/h264_decoder.c b/dlls/winegstreamer/h264_decoder.c index 041421a427d..bcb9f341aaa 100644 --- a/dlls/winegstreamer/h264_decoder.c +++ b/dlls/winegstreamer/h264_decoder.c @@ -75,9 +75,15 @@ static struct h264_decoder *impl_from_IMFTransform(IMFTransform *iface)
static HRESULT try_create_wg_transform(struct h264_decoder *decoder) { + /* Call of Duty: Black Ops 3 doesn't care about the ProcessInput/ProcessOutput + * return values, it calls them in a specific order and expects the decoder + * transform to be able to queue its input buffers. We need to use a buffer list + * to match its expectations. + */ struct wg_transform_attrs attrs = { .output_plane_align = 15, + .input_queue_length = 15, }; struct wg_format input_format; struct wg_format output_format; diff --git a/dlls/winegstreamer/unixlib.h b/dlls/winegstreamer/unixlib.h index 15b5dcff352..938319faea7 100644 --- a/dlls/winegstreamer/unixlib.h +++ b/dlls/winegstreamer/unixlib.h @@ -308,6 +308,7 @@ struct wg_parser_stream_seek_params struct wg_transform_attrs { UINT32 output_plane_align; + UINT32 input_queue_length; };
struct wg_transform_create_params diff --git a/dlls/winegstreamer/wg_transform.c b/dlls/winegstreamer/wg_transform.c index 64310747d82..4ea68dd5a75 100644 --- a/dlls/winegstreamer/wg_transform.c +++ b/dlls/winegstreamer/wg_transform.c @@ -51,7 +51,6 @@ struct wg_transform GstSegment segment; GstQuery *drain_query;
- guint input_max_length; GstAtomicQueue *input_queue;
bool input_is_flipped; @@ -307,7 +306,6 @@ NTSTATUS wg_transform_create(void *args) if (!(transform->allocator = wg_allocator_create(transform_request_sample, transform))) goto out; transform->attrs = *params->attrs; - transform->input_max_length = 1; transform->output_format = output_format;
/* We need to use GST_FORMAT_TIME here because it's the only format @@ -348,12 +346,6 @@ NTSTATUS wg_transform_create(void *args) switch (input_format.major_type) { case WG_MAJOR_TYPE_VIDEO_H264: - /* Call of Duty: Black Ops 3 doesn't care about the ProcessInput/ProcessOutput - * return values, it calls them in a specific order and expects the decoder - * transform to be able to queue its input buffers. We need to use a buffer list - * to match its expectations. - */ - transform->input_max_length = 16; if (!(element = create_element("h264parse", "base")) || !append_element(transform->container, element, &first, &last)) goto out; @@ -567,7 +559,7 @@ NTSTATUS wg_transform_push_data(void *args) guint length;
length = gst_atomic_queue_length(transform->input_queue); - if (length >= transform->input_max_length) + if (length >= transform->attrs.input_queue_length + 1) { GST_INFO("Refusing %u bytes, %u buffers already queued", sample->size, length); params->result = MF_E_NOTACCEPTING; @@ -868,7 +860,7 @@ NTSTATUS wg_transform_get_status(void *args) struct wg_transform_get_status_params *params = args; struct wg_transform *transform = params->transform;
- params->accepts_input = gst_atomic_queue_length(transform->input_queue) < transform->input_max_length; + params->accepts_input = gst_atomic_queue_length(transform->input_queue) < transform->attrs.input_queue_length + 1; return STATUS_SUCCESS; }
From: Rémi Bernon rbernon@codeweavers.com
--- dlls/mf/tests/transform.c | 12 ++---------- dlls/winegstreamer/h264_decoder.c | 4 ++++ dlls/winegstreamer/unixlib.h | 1 + dlls/winegstreamer/wg_transform.c | 23 +++++++++++++++++++++++ 4 files changed, 30 insertions(+), 10 deletions(-)
diff --git a/dlls/mf/tests/transform.c b/dlls/mf/tests/transform.c index a8e1cc83c62..168c7afb43c 100644 --- a/dlls/mf/tests/transform.c +++ b/dlls/mf/tests/transform.c @@ -4026,7 +4026,6 @@ static void test_h264_decoder_concat_streams(void) const struct buffer_desc output_buffer_desc[] = { {.length = 0x3600}, - {.length = 0x3600, .todo_length = TRUE}, {.length = 0x4980}, }; const struct attribute_desc output_sample_attributes[] = @@ -4039,18 +4038,12 @@ static void test_h264_decoder_concat_streams(void) { .attributes = output_sample_attributes + 0, .sample_time = 0, .sample_duration = 333667, - .buffer_count = 1, .buffers = output_buffer_desc + 0, .repeat_count = 57, - }, - { - .attributes = output_sample_attributes + 0, - .sample_time = 19352666, .sample_duration = 333667, - .buffer_count = 1, .buffers = output_buffer_desc + 1, .repeat_count = 1, - .todo_length = TRUE, + .buffer_count = 1, .buffers = output_buffer_desc + 0, .repeat_count = 59, }, { .attributes = output_sample_attributes + 0, .sample_time = 60 * 333667, .sample_duration = 333667, - .buffer_count = 1, .buffers = output_buffer_desc + 2, .repeat_count = 59, + .buffer_count = 1, .buffers = output_buffer_desc + 1, .repeat_count = 59, }, };
@@ -4150,7 +4143,6 @@ static void test_h264_decoder_concat_streams(void)
hr = IMFCollection_GetElementCount(output_samples, &count); ok(hr == S_OK, "GetElementCount returned %#lx\n", hr); - todo_wine ok(count >= 0x77, "GetElementCount returned %#lx\n", count);
ret = check_mf_sample_collection(output_samples, output_sample_desc, NULL); diff --git a/dlls/winegstreamer/h264_decoder.c b/dlls/winegstreamer/h264_decoder.c index bcb9f341aaa..ab4d883d0df 100644 --- a/dlls/winegstreamer/h264_decoder.c +++ b/dlls/winegstreamer/h264_decoder.c @@ -87,6 +87,7 @@ static HRESULT try_create_wg_transform(struct h264_decoder *decoder) }; struct wg_format input_format; struct wg_format output_format; + UINT32 low_latency;
if (decoder->wg_transform) wg_transform_destroy(decoder->wg_transform); @@ -108,6 +109,9 @@ static HRESULT try_create_wg_transform(struct h264_decoder *decoder) output_format.u.video.fps_d = 0; output_format.u.video.fps_n = 0;
+ if (SUCCEEDED(IMFAttributes_GetUINT32(decoder->attributes, &MF_LOW_LATENCY, &low_latency))) + attrs.low_latency = !!low_latency; + if (!(decoder->wg_transform = wg_transform_create(&input_format, &output_format, &attrs))) return E_FAIL;
diff --git a/dlls/winegstreamer/unixlib.h b/dlls/winegstreamer/unixlib.h index 938319faea7..a9892bf9d4f 100644 --- a/dlls/winegstreamer/unixlib.h +++ b/dlls/winegstreamer/unixlib.h @@ -309,6 +309,7 @@ struct wg_transform_attrs { UINT32 output_plane_align; UINT32 input_queue_length; + BOOL low_latency; };
struct wg_transform_create_params diff --git a/dlls/winegstreamer/wg_transform.c b/dlls/winegstreamer/wg_transform.c index 4ea68dd5a75..76dbc31ff74 100644 --- a/dlls/winegstreamer/wg_transform.c +++ b/dlls/winegstreamer/wg_transform.c @@ -106,6 +106,26 @@ static GstFlowReturn transform_sink_chain_cb(GstPad *pad, GstObject *parent, Gst return GST_FLOW_OK; }
+static gboolean transform_src_query_latency(struct wg_transform *transform, GstQuery *query) +{ + GST_LOG("transform %p, query %p", transform, query); + gst_query_set_latency(query, transform->attrs.low_latency, 0, 0); + return true; +} + +static gboolean transform_src_query_cb(GstPad *pad, GstObject *parent, GstQuery *query) +{ + struct wg_transform *transform = gst_pad_get_element_private(pad); + + switch (query->type) + { + case GST_QUERY_LATENCY: + return transform_src_query_latency(transform, query); + default: + return gst_pad_query_default(pad, parent, query); + } +} + static gboolean transform_sink_query_cb(GstPad *pad, GstObject *parent, GstQuery *query) { struct wg_transform *transform = gst_pad_get_element_private(pad); @@ -321,6 +341,9 @@ NTSTATUS wg_transform_create(void *args) if (!transform->my_src) goto out;
+ gst_pad_set_element_private(transform->my_src, transform); + gst_pad_set_query_function(transform->my_src, transform_src_query_cb); + if (!(transform->output_caps = wg_format_to_caps(&output_format))) goto out; if (!(template = gst_pad_template_new("sink", GST_PAD_SINK, GST_PAD_ALWAYS, transform->output_caps)))
Hi,
It looks like your patch introduced the new failures shown below. Please investigate and fix them before resubmitting your patch. If they are not new, fixing them anyway would help a lot. Otherwise please ask for the known failures list to be updated.
The tests also ran into some preexisting test failures. If you know how to fix them that would be helpful. See the TestBot job for the details:
The full results can be found at: https://testbot.winehq.org/JobDetails.pl?Key=133019
Your paranoid android.
=== debian11 (32 bit zh:CN report) ===
mf: mf.c:5055: Test failed: WaitForSingleObject returned 258 mf.c:5055: Test failed: Unexpected hr 0xd36d8. Unhandled exception: page fault on execute access to 0x00000000 in 32-bit code (0x00000000).
@gofman As we've discussed that you had some unique failures on your distro, would you mind checking if this works? I've tried with every combination of plugins I could try (vaapi, avdec, openh264) and they are all passing the tests (there's a couple of differences, but minor things).
Yeah, without extra events we discussed before it didn't ever work here with libva... I will check again and suggest a change which works.
On Wed May 24 16:40:08 2023 +0000, Paul Gofman wrote:
Yeah, without extra events we discussed before it didn't ever work here with libva... I will check again and suggest a change which works.
This is slightly different from what is in Proton. I've added `eos` / `stream_start` events which seem to be the thing that works best with all the plugins I've tried. `flush_start` / `flush_stop` doesn't work well for me with VA-API decoder.
On Wed May 24 16:40:21 2023 +0000, Rémi Bernon wrote:
This is slightly different from what is in Proton. I've added `eos` / `stream_start` events which seem to be the thing that works best with all the plugins I've tried. `flush_start` / `flush_stop` doesn't work well for me with VA-API decoder.
I tried EOS / steam start back then and it didn't work for me even in Proton as well as with local libva (that is, drain was performed this way but stream couldn't continue). But I will check, maybe something is different with these patches.
On Wed May 24 16:42:58 2023 +0000, Paul Gofman wrote:
I tried EOS / steam start back then and it didn't work for me even in Proton as well as with local libva (that is, drain was performed this way but stream couldn't continue). But I will check, maybe something is different with these patches.
It could also have been because `vaapidecodebin` is and was never working well with `wg_transform`. And I've added https://gitlab.winehq.org/wine/wine/-/merge_requests/2893/diffs?commit_id=42... to workaround that.
In the future, can you please submit smaller series? 5 patches at a time is more manageable.
Zebediah Figura (@zfigura) commented about dlls/winegstreamer/unixlib.c:
for (tmp = transforms; tmp != NULL && element == NULL; tmp = tmp->next) { name = gst_plugin_feature_get_name(GST_PLUGIN_FEATURE(tmp->data));
if (!strcmp(name, "vaapidecodebin")){/* vaapidecodebin adds too much abstraction and wg_transform works best with lower-level elements. */
There may be a good reason to blacklist vaapidecodebin, but I don't think this is specific enough. What about vaapidecodebin is "too much abstraction"?
They are sometimes harmless, for instance when using openh264 decoder.
Wait, what? What kind of errors are harmless?
On Wed May 24 18:19:04 2023 +0000, Zebediah Figura wrote:
There may be a good reason to blacklist vaapidecodebin, but I don't think this is specific enough. What about vaapidecodebin is "too much abstraction"?
Rémi may probably answer with more facts and issues with decodebin, but what I observed previously vaadecodebin ignores low-latency parameter while the actual vaapi h264 decoder minds that.
On Wed May 24 18:26:38 2023 +0000, Zebediah Figura wrote:
They are sometimes harmless, for instance when using openh264 decoder.
Wait, what? What kind of errors are harmless?
The openh264 decoder sometimes returns GST_FLOW_NOT_NEGOTIATED (-4) although it just works fine if we ignore it and keep pushing data. It happens for instance when running the tests.
On Wed May 24 18:24:39 2023 +0000, Paul Gofman wrote:
Rémi may probably answer with more facts and issues with decodebin, but what I observed previously vaadecodebin ignores low-latency parameter while the actual vaapi h264 decoder minds that.
It's a sub-pipeline and it adds all sort of complexity that we don't want to bother with for wg_transform. Threading being one major issue as we want as much control as possible over the processing of samples, ideally making it synchronously.
The wg_transform interface is (ideally) meant to be a synchronous wrapper over decoders / converters, with all the threading problems moved over the user side where we control every detail.
Patch 5/10 doesn't make much sense by itself. Looking ahead, it seems to be used to send segment-done events, but this seems like an unclear way to achieve that—clearer would be to explicitly store the last pts separately and just use that.
However, stepping back, the segment-done logic is concerning. GStreamer is annoyingly underdocumented here, and it's not clear to me that it's actually legal to send segment-done with a different timestamp than the previous segment event. Moreover, though, it's not actually clear to me why we need to?
On Wed May 24 18:44:37 2023 +0000, Zebediah Figura wrote:
Patch 5/10 doesn't make much sense by itself. Looking ahead, it seems to be used to send segment-done events, but this seems like an unclear way to achieve that—clearer would be to explicitly store the last pts separately and just use that. However, stepping back, the segment-done logic is concerning. GStreamer is annoyingly underdocumented here, and it's not clear to me that it's actually legal to send segment-done with a different timestamp than the previous segment event. Moreover, though, it's not actually clear to me why we need to?
We don't have to send the segment end timestamp, and it works just the same if we use -1 like for when it was started. What matters it to restore the timestamp for stream start so that buffer times are kept consistent.
I don't see how it is different to keep the timestamp in a field vs keep the segment. I prefer keeping the segment as it saves useless local variable (and segment initialization every time we want to create one).
On Wed May 24 17:08:10 2023 +0000, Rémi Bernon wrote:
It could also have been because `vaapidecodebin` is and was never working well with `wg_transform`. And I've added https://gitlab.winehq.org/wine/wine/-/merge_requests/2893/diffs?commit_id=42... to workaround that.
I've checked (with both the tests in the patchset and a bit different ones I had on the matter) and it seems to work here.
There is an unrelated failure with my older test though from Proton patches. That is not about flushing the pipeline but checking sample size before querying the data from wg_transform, which check is wrong because the call might not need to output a sample at all or maybe return a sample of a smaller size. I am attaching the test and a Proton patch which is fixing that (rebased on top of the present MR). This is apparently a different aspect orthogonal to what this MR, attaching that for future reference only.
[0001-mf-tests-Add-a-test-with-h264-reuse-with-a-different.patch](/uploads/dc71eb65faa61f463f0cccc5540587fd/0001-mf-tests-Add-a-test-with-h264-reuse-with-a-different.patch)
[0002-winegstreamer-Don-t-pre-check-sample-size-in-wg_tran.patch](/uploads/581fe905c6ca3e4f1c11eaf162d5ef10/0002-winegstreamer-Don-t-pre-check-sample-size-in-wg_tran.patch)
On Wed May 24 19:28:47 2023 +0000, Paul Gofman wrote:
I've checked (with both the tests in the patchset and a bit different ones I had on the matter) and it seems to work here. There is an unrelated failure with my older test though from Proton patches. That is not about flushing the pipeline but checking sample size before querying the data from wg_transform, which check is wrong because the call might not need to output a sample at all or maybe return a sample of a smaller size. I am attaching the test and a Proton patch which is fixing that (rebased on top of the present MR). This is apparently a different aspect orthogonal to what this MR, attaching that for future reference only. [0001-mf-tests-Add-a-test-with-h264-reuse-with-a-different.patch](/uploads/dc71eb65faa61f463f0cccc5540587fd/0001-mf-tests-Add-a-test-with-h264-reuse-with-a-different.patch) [0002-winegstreamer-Don-t-pre-check-sample-size-in-wg_tran.patch](/uploads/581fe905c6ca3e4f1c11eaf162d5ef10/0002-winegstreamer-Don-t-pre-check-sample-size-in-wg_tran.patch)
Thanks, I've included a similar test into the concatenated stream test. I believe the problem only happens before the first stream type change, when we don't know yet what size the buffer should be.
I've split these into https://gitlab.winehq.org/wine/wine/-/merge_requests/2901, which I think should come before this MR, as generating the timestamps ourselves also solve the mismatches that drain and flush are sometimes causing.
It doesn't look like Windows really respect the stream embedded information that h264parse generates, and once we do it ourselves it becomes useless, so I've removed the element and it then works pretty much like Proton.
-
Marvin -
Paul Gofman -
Paul Gofman (＠gofman)
-
Rémi Bernon
-
Rémi Bernon (＠rbernon)
-
Zebediah Figura (＠zfigura)