[PATCH v10 0/5] MR7623: winegstreamer: Use provided PTS and duration in video_decoder.
This MR modifies winegstreamer to match Windows behaviour in that: 1. the video decoders will output the PTS and duration of the input sample (if provided); and 2. the WMV decoder will set any value not provided to zero It also adds support for supplying a DTS value to the MFTs. I've marked this as draft as it fixes the tests in MR !7563 (in addition to fixing some existing `test_wmv_decoder` tests). Also, as demonstrated in MR !7569, our demuxers output different timestamps to Windows. This change will result in those different timestamps being forwarded from the decoder. So we may also want to address that difference prior to accepting this MR. -- v10: winegstreamer: Fixup negative input timestamps. winegstreamer: Don't generate sample timestamps for the WMV decoder. winegstreamer: Avoid rounding errors when generating timestamps. winegstreamer: Use provided PTS and duration in video_decoder. mf/tests: Add negative timestamp tests for h264. https://gitlab.winehq.org/wine/wine/-/merge_requests/7623
From: Brendan McGrath <bmcgrath(a)codeweavers.com> --- dlls/mf/tests/transform.c | 118 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 118 insertions(+) diff --git a/dlls/mf/tests/transform.c b/dlls/mf/tests/transform.c index dfe3bddd454..2ac7906eda6 100644 --- a/dlls/mf/tests/transform.c +++ b/dlls/mf/tests/transform.c @@ -5195,6 +5195,58 @@ static void test_h264_decoder_timestamps(void) { 4003999, 333667 }, { 4337666, 333666 }, }; + static const struct timestamps input_sample_neg_ts[] = + { + { -333667, 333667, -1001000 }, + { -333667, 333667, -1001000 }, + { -333667, 333667, -1001000 }, + { 1000999, 333667, -667334 }, + { 333666, 333666, -333667 }, + { 0, 333666 }, + { 667332, 333667, 333666 }, + { 2335666, 333667, 667333 }, + { 1668333, 333666, 1001000 }, + { 1334666, 333667 }, + { 2001999, 333667, 1668333 }, + { 3670333, 333667, 2002000 }, + { 3002999, 333667, 2335666 }, + { 2669333, 333666 }, + { 3336666, 333667, 3003000 }, + { 5004999, 333667, 3336666 }, + { 4337666, 333666, 3670333 }, + { 4004000, 333666 }, + { 4671332, 333667, 4337666 }, + { 6339666, 333667, 4671333 }, + { 5672333, 333666, 5005000 }, + { 5338666, 333667 }, + { 6005999, 333667, 5672333 }, + { 7674333, 333667, 6006000 }, + { 7006999, 333667, 6339666 }, + { 6673333, 333666 }, + { 7340666, 333667, 7007000 }, + { 9008999, 333667, 7340666 }, + { 8341666, 333666, 7674333 }, + { 8008000, 333666 }, + { 8675332, 333667, 8341666 }, + { 10343666, 333667, 8675333 }, + { 9676333, 333666, 9009000 }, + { 9342666, 333667 }, + { 10009999, 333667, 9676333 }, + { 11678333, 333667, 10010000 }, + }; + static const struct timestamps exp_sample_neg_ts[] = + { + { -333667, 333667 }, + { 0, 333666 }, + { 333666, 333666 }, + { 667332, 333667 }, + { 1000999, 333667 }, + { 1334666, 333667 }, + { 1668333, 333666 }, + { 2001999, 333667 }, + { 2335666, 333667 }, + { 2669333, 333666 }, + }; IMFSample *input_sample, *output_sample; const BYTE *h264_encoded_data; @@ -5376,6 +5428,72 @@ static void test_h264_decoder_timestamps(void) ret = IMFTransform_Release(transform); ok(ret == 0, "Release returned %lu\n", ret); + /* Test when supplied sample timestamps include negative values */ + hr = CoCreateInstance(&CLSID_MSH264DecoderMFT, NULL, CLSCTX_INPROC_SERVER, + &IID_IMFTransform, (void **)&transform); + ok(hr == S_OK, "Got %#lx\n", hr); + + load_resource(L"h264data.bin", &h264_encoded_data, &h264_encoded_data_len); + + check_mft_set_input_type(transform, input_type_24fps_desc, S_OK); + check_mft_set_output_type(transform, output_type_24fps_desc, S_OK); + + hr = IMFTransform_ProcessMessage(transform, MFT_MESSAGE_NOTIFY_START_OF_STREAM, 0); + ok(hr == S_OK, "Got %#lx\n", hr); + + j = 0; + output_sample = create_sample(NULL, actual_width * actual_height * 2); + for (i = 0; i < ARRAYSIZE(exp_sample_neg_ts);) + { + winetest_push_context("output %ld", i); + hr = check_mft_process_output(transform, output_sample, &output_status); + ok(hr == S_OK || hr == MF_E_TRANSFORM_STREAM_CHANGE || hr == MF_E_TRANSFORM_NEED_MORE_INPUT, "ProcessOutput returned %#lx\n", hr); + if (hr == S_OK) + { + hr = IMFSample_GetSampleTime(output_sample, &time); + ok(hr == S_OK, "Got %#lx\n", hr); + todo_wine_if(i) + ok(time == exp_sample_neg_ts[i].time, "got time %I64d, expected %I64d\n", time, exp_sample_neg_ts[i].time); + hr = IMFSample_GetSampleDuration(output_sample, &duration); + ok(hr == S_OK, "Got %#lx\n", hr); + todo_wine + ok(duration == exp_sample_neg_ts[i].duration, "got duration %I64d, expected %I64d\n", duration, exp_sample_neg_ts[i].duration); + ret = IMFSample_Release(output_sample); + ok(ret == 0, "Release returned %lu\n", ret); + output_sample = create_sample(NULL, actual_width * actual_height * 2); + i++; + } + else if (hr == MF_E_TRANSFORM_NEED_MORE_INPUT && j < ARRAYSIZE(input_sample_neg_ts)) + { + input_sample = next_h264_sample(&h264_encoded_data, &h264_encoded_data_len); + hr = IMFSample_SetSampleTime(input_sample, input_sample_neg_ts[j].time); + ok(hr == S_OK, "Got %#lx\n", hr); + hr = IMFSample_SetSampleDuration(input_sample, input_sample_neg_ts[j].duration); + ok(hr == S_OK, "Got %#lx\n", hr); + if (input_sample_neg_ts[j].dts) + { + hr = IMFSample_SetUINT64(input_sample, &MFSampleExtension_DecodeTimestamp, input_sample_neg_ts[j].dts); + ok(hr == S_OK, "Got %#lx\n", hr); + } + j++; + hr = IMFTransform_ProcessInput(transform, 0, input_sample, 0); + ok(hr == S_OK, "ProcessInput returned %#lx\n", hr); + ret = IMFSample_Release(input_sample); + ok(ret <= 1, "Release returned %lu\n", ret); + } + else if (hr == MF_E_TRANSFORM_NEED_MORE_INPUT && j == ARRAYSIZE(input_sample_neg_ts)) + { + ok(0, "no more input to provide\n"); + i = ARRAYSIZE(exp_sample_neg_ts); + } + winetest_pop_context(); + } + + ret = IMFSample_Release(output_sample); + ok(ret == 0, "Release returned %lu\n", ret); + ret = IMFTransform_Release(transform); + ok(ret == 0, "Release returned %lu\n", ret); + failed: winetest_pop_context(); CoUninitialize(); -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/7623
From: Brendan McGrath <bmcgrath(a)codeweavers.com> --- dlls/mf/tests/transform.c | 33 ++++-------------- dlls/winegstreamer/aac_decoder.c | 2 +- dlls/winegstreamer/color_convert.c | 2 +- dlls/winegstreamer/gst_private.h | 2 +- dlls/winegstreamer/resampler.c | 2 +- dlls/winegstreamer/unixlib.h | 2 ++ dlls/winegstreamer/video_decoder.c | 25 +++++++------ dlls/winegstreamer/video_encoder.c | 2 +- dlls/winegstreamer/video_processor.c | 2 +- dlls/winegstreamer/wg_sample.c | 4 ++- dlls/winegstreamer/wg_transform.c | 52 ++++++++++++++++++++++------ dlls/winegstreamer/wma_decoder.c | 2 +- 12 files changed, 74 insertions(+), 56 deletions(-) diff --git a/dlls/mf/tests/transform.c b/dlls/mf/tests/transform.c index 2ac7906eda6..d450c4aecec 100644 --- a/dlls/mf/tests/transform.c +++ b/dlls/mf/tests/transform.c @@ -5386,11 +5386,9 @@ static void test_h264_decoder_timestamps(void) { hr = IMFSample_GetSampleTime(output_sample, &time); ok(hr == S_OK, "Got %#lx\n", hr); - todo_wine_if(i) ok(time == exp_sample_ts[i].time, "got time %I64d, expected %I64d\n", time, exp_sample_ts[i].time); hr = IMFSample_GetSampleDuration(output_sample, &duration); ok(hr == S_OK, "Got %#lx\n", hr); - todo_wine ok(duration == exp_sample_ts[i].duration, "got duration %I64d, expected %I64d\n", duration, exp_sample_ts[i].duration); ret = IMFSample_Release(output_sample); ok(ret == 0, "Release returned %lu\n", ret); @@ -5452,11 +5450,10 @@ static void test_h264_decoder_timestamps(void) { hr = IMFSample_GetSampleTime(output_sample, &time); ok(hr == S_OK, "Got %#lx\n", hr); - todo_wine_if(i) + todo_wine_if(exp_sample_neg_ts[i].time < 0) ok(time == exp_sample_neg_ts[i].time, "got time %I64d, expected %I64d\n", time, exp_sample_neg_ts[i].time); hr = IMFSample_GetSampleDuration(output_sample, &duration); ok(hr == S_OK, "Got %#lx\n", hr); - todo_wine ok(duration == exp_sample_neg_ts[i].duration, "got duration %I64d, expected %I64d\n", duration, exp_sample_neg_ts[i].duration); ret = IMFSample_Release(output_sample); ok(ret == 0, "Release returned %lu\n", ret); @@ -6835,28 +6832,12 @@ static void test_wmv_decoder(void) .attributes = output_sample_attributes, .sample_time = 0, .sample_duration = 333333, .buffer_count = 1, .buffers = &output_buffer_desc_nv12, - .todo_duration = TRUE, - }; - const struct sample_desc output_sample_desc_nv12_todo_time = - { - .attributes = output_sample_attributes, - .sample_time = 0, .sample_duration = 333333, - .buffer_count = 1, .buffers = &output_buffer_desc_nv12, - .todo_time = TRUE, .todo_duration = TRUE, }; const struct sample_desc output_sample_desc_rgb = { .attributes = output_sample_attributes, .sample_time = 0, .sample_duration = 333333, .buffer_count = 1, .buffers = &output_buffer_desc_rgb, - .todo_duration = TRUE, - }; - const struct sample_desc output_sample_desc_rgb_todo_time = - { - .attributes = output_sample_attributes, - .sample_time = 0, .sample_duration = 333333, - .buffer_count = 1, .buffers = &output_buffer_desc_rgb, - .todo_time = TRUE, .todo_duration = TRUE, }; const struct transform_desc @@ -6890,7 +6871,7 @@ static void test_wmv_decoder(void) .expect_output_type_desc = expect_output_type_desc, .expect_input_info = &expect_input_info, .expect_output_info = &expect_output_info, - .output_sample_desc = &output_sample_desc_nv12_todo_time, + .output_sample_desc = &output_sample_desc_nv12, .result_bitmap = L"nv12frame.bmp", .delta = 0, }, @@ -6901,7 +6882,7 @@ static void test_wmv_decoder(void) .expect_output_type_desc = expect_output_type_desc_rgb, .expect_input_info = &expect_input_info_rgb, .expect_output_info = &expect_output_info_rgb, - .output_sample_desc = &output_sample_desc_rgb_todo_time, + .output_sample_desc = &output_sample_desc_rgb, .result_bitmap = L"rgb32frame-flip.bmp", .delta = 5, }, @@ -6912,7 +6893,7 @@ static void test_wmv_decoder(void) .expect_output_type_desc = expect_output_type_desc_rgb_negative_stride, .expect_input_info = &expect_input_info_rgb, .expect_output_info = &expect_output_info_rgb, - .output_sample_desc = &output_sample_desc_rgb_todo_time, + .output_sample_desc = &output_sample_desc_rgb, .result_bitmap = L"rgb32frame-flip.bmp", .delta = 5, }, @@ -6923,7 +6904,7 @@ static void test_wmv_decoder(void) .expect_output_type_desc = expect_output_type_desc_rgb, .expect_input_info = &expect_input_info_rgb, .expect_output_info = &expect_output_info_rgb, - .output_sample_desc = &output_sample_desc_rgb_todo_time, + .output_sample_desc = &output_sample_desc_rgb, .result_bitmap = L"rgb32frame-flip.bmp", .delta = 5, }, @@ -6946,7 +6927,7 @@ static void test_wmv_decoder(void) .expect_output_type_desc = expect_output_type_desc_rgb_negative_stride, .expect_input_info = &expect_input_info_rgb, .expect_output_info = &expect_output_info_rgb, - .output_sample_desc = &output_sample_desc_rgb_todo_time, + .output_sample_desc = &output_sample_desc_rgb, .result_bitmap = L"rgb32frame.bmp", .delta = 5, }, @@ -7423,11 +7404,9 @@ static void test_wmv_decoder_timestamps(void) { hr = IMFSample_GetSampleTime(output_sample, &time); ok(hr == S_OK, "Got %#lx\n", hr); - todo_wine_if(i) ok(time == exp_sample_ts[i].time, "got time %I64d, expected %I64d\n", time, exp_sample_ts[i].time); hr = IMFSample_GetSampleDuration(output_sample, &duration); ok(hr == S_OK, "Got %#lx\n", hr); - todo_wine ok(duration == exp_sample_ts[i].duration, "got duration %I64d, expected %I64d\n", duration, exp_sample_ts[i].duration); ret = IMFSample_Release(output_sample); ok(ret == 0, "Release returned %lu\n", ret); diff --git a/dlls/winegstreamer/aac_decoder.c b/dlls/winegstreamer/aac_decoder.c index cf5b47edf68..727c516f044 100644 --- a/dlls/winegstreamer/aac_decoder.c +++ b/dlls/winegstreamer/aac_decoder.c @@ -563,7 +563,7 @@ static HRESULT WINAPI transform_ProcessOutput(IMFTransform *iface, DWORD flags, return hr; if (SUCCEEDED(hr = wg_transform_read_mf(decoder->wg_transform, samples->pSample, - info.cbSize, &samples->dwStatus))) + info.cbSize, &samples->dwStatus, NULL))) wg_sample_queue_flush(decoder->wg_sample_queue, false); else samples->dwStatus = MFT_OUTPUT_DATA_BUFFER_NO_SAMPLE; diff --git a/dlls/winegstreamer/color_convert.c b/dlls/winegstreamer/color_convert.c index e466a9289b1..c5c7141f4ce 100644 --- a/dlls/winegstreamer/color_convert.c +++ b/dlls/winegstreamer/color_convert.c @@ -670,7 +670,7 @@ static HRESULT WINAPI transform_ProcessOutput(IMFTransform *iface, DWORD flags, return hr; if (SUCCEEDED(hr = wg_transform_read_mf(impl->wg_transform, samples->pSample, - info.cbSize, &samples->dwStatus))) + info.cbSize, &samples->dwStatus, NULL))) wg_sample_queue_flush(impl->wg_sample_queue, false); return hr; diff --git a/dlls/winegstreamer/gst_private.h b/dlls/winegstreamer/gst_private.h index 9db484c033f..60e0c3af8bc 100644 --- a/dlls/winegstreamer/gst_private.h +++ b/dlls/winegstreamer/gst_private.h @@ -167,7 +167,7 @@ HRESULT wg_transform_push_quartz(wg_transform_t transform, struct wg_sample *sam HRESULT wg_transform_push_dmo(wg_transform_t transform, IMediaBuffer *media_buffer, DWORD flags, REFERENCE_TIME time_stamp, REFERENCE_TIME time_length, struct wg_sample_queue *queue); HRESULT wg_transform_read_mf(wg_transform_t transform, IMFSample *sample, - DWORD sample_size, DWORD *flags); + DWORD sample_size, DWORD *flags, bool *preserve_timestamps); HRESULT wg_transform_read_quartz(wg_transform_t transform, struct wg_sample *sample); HRESULT wg_transform_read_dmo(wg_transform_t transform, DMO_OUTPUT_DATA_BUFFER *buffer); diff --git a/dlls/winegstreamer/resampler.c b/dlls/winegstreamer/resampler.c index c1ce9897ef1..5e38243ac2b 100644 --- a/dlls/winegstreamer/resampler.c +++ b/dlls/winegstreamer/resampler.c @@ -541,7 +541,7 @@ static HRESULT WINAPI transform_ProcessOutput(IMFTransform *iface, DWORD flags, return hr; if (SUCCEEDED(hr = wg_transform_read_mf(impl->wg_transform, samples->pSample, - info.cbSize, &samples->dwStatus))) + info.cbSize, &samples->dwStatus, NULL))) wg_sample_queue_flush(impl->wg_sample_queue, false); return hr; diff --git a/dlls/winegstreamer/unixlib.h b/dlls/winegstreamer/unixlib.h index 19270bd731b..d1a414b6396 100644 --- a/dlls/winegstreamer/unixlib.h +++ b/dlls/winegstreamer/unixlib.h @@ -179,6 +179,7 @@ enum wg_sample_flag WG_SAMPLE_FLAG_HAS_DURATION = 4, WG_SAMPLE_FLAG_SYNC_POINT = 8, WG_SAMPLE_FLAG_DISCONTINUITY = 0x10, + WG_SAMPLE_FLAG_PRESERVE_TIMESTAMPS = 0x20, }; struct wg_sample @@ -335,6 +336,7 @@ struct wg_transform_attrs UINT32 input_queue_length; BOOL allow_format_change; BOOL low_latency; + BOOL preserve_timestamps; }; struct wg_transform_create_params diff --git a/dlls/winegstreamer/video_decoder.c b/dlls/winegstreamer/video_decoder.c index 4086b586d76..47b2eb0371e 100644 --- a/dlls/winegstreamer/video_decoder.c +++ b/dlls/winegstreamer/video_decoder.c @@ -944,6 +944,7 @@ static HRESULT WINAPI transform_ProcessOutput(IMFTransform *iface, DWORD flags, LONGLONG duration; IMFSample *sample; UINT64 frame_size, frame_rate; + bool preserve_timestamps; GUID subtype; DWORD size; HRESULT hr; @@ -989,19 +990,22 @@ static HRESULT WINAPI transform_ProcessOutput(IMFTransform *iface, DWORD flags, } if (SUCCEEDED(hr = wg_transform_read_mf(decoder->wg_transform, sample, - sample_size, &samples->dwStatus))) + sample_size, &samples->dwStatus, &preserve_timestamps))) { wg_sample_queue_flush(decoder->wg_sample_queue, false); - if (FAILED(IMFMediaType_GetUINT64(decoder->input_type, &MF_MT_FRAME_RATE, &frame_rate))) - frame_rate = (UINT64)30000 << 32 | 1001; - - duration = (UINT64)10000000 * (UINT32)frame_rate / (frame_rate >> 32); - if (FAILED(IMFSample_SetSampleTime(sample, decoder->sample_time))) - WARN("Failed to set sample time\n"); - if (FAILED(IMFSample_SetSampleDuration(sample, duration))) - WARN("Failed to set sample duration\n"); - decoder->sample_time += duration; + if (!preserve_timestamps) + { + if (FAILED(IMFMediaType_GetUINT64(decoder->input_type, &MF_MT_FRAME_RATE, &frame_rate))) + frame_rate = (UINT64)30000 << 32 | 1001; + + duration = (UINT64)10000000 * (UINT32)frame_rate / (frame_rate >> 32); + if (FAILED(IMFSample_SetSampleTime(sample, decoder->sample_time))) + WARN("Failed to set sample time\n"); + if (FAILED(IMFSample_SetSampleDuration(sample, duration))) + WARN("Failed to set sample duration\n"); + decoder->sample_time += duration; + } } if (hr == MF_E_TRANSFORM_STREAM_CHANGE) @@ -1633,6 +1637,7 @@ static HRESULT video_decoder_create_with_types(const GUID *const *input_types, U goto failed; decoder->wg_transform_attrs.input_queue_length = 15; + decoder->wg_transform_attrs.preserve_timestamps = TRUE; *out = decoder; TRACE("Created decoder %p\n", decoder); diff --git a/dlls/winegstreamer/video_encoder.c b/dlls/winegstreamer/video_encoder.c index fd28cd150da..f26bec66c84 100644 --- a/dlls/winegstreamer/video_encoder.c +++ b/dlls/winegstreamer/video_encoder.c @@ -541,7 +541,7 @@ static HRESULT WINAPI transform_ProcessOutput(IMFTransform *iface, DWORD flags, if (!samples->pSample) return E_INVALIDARG; - if (SUCCEEDED(hr = wg_transform_read_mf(encoder->wg_transform, samples->pSample, 0, &samples->dwStatus))) + if (SUCCEEDED(hr = wg_transform_read_mf(encoder->wg_transform, samples->pSample, 0, &samples->dwStatus, NULL))) wg_sample_queue_flush(encoder->wg_sample_queue, false); return hr; diff --git a/dlls/winegstreamer/video_processor.c b/dlls/winegstreamer/video_processor.c index 5f270c13550..3e327228e65 100644 --- a/dlls/winegstreamer/video_processor.c +++ b/dlls/winegstreamer/video_processor.c @@ -731,7 +731,7 @@ static HRESULT WINAPI video_processor_ProcessOutput(IMFTransform *iface, DWORD f IMFSample_AddRef(output_sample); } - if (FAILED(hr = wg_transform_read_mf(impl->wg_transform, output_sample, info.cbSize, &samples->dwStatus))) + if (FAILED(hr = wg_transform_read_mf(impl->wg_transform, output_sample, info.cbSize, &samples->dwStatus, NULL))) goto done; wg_sample_queue_flush(impl->wg_sample_queue, false); diff --git a/dlls/winegstreamer/wg_sample.c b/dlls/winegstreamer/wg_sample.c index 116dbb1f3ec..83f7163d0c9 100644 --- a/dlls/winegstreamer/wg_sample.c +++ b/dlls/winegstreamer/wg_sample.c @@ -339,7 +339,7 @@ HRESULT wg_transform_push_mf(wg_transform_t transform, IMFSample *sample, } HRESULT wg_transform_read_mf(wg_transform_t transform, IMFSample *sample, - DWORD sample_size, DWORD *flags) + DWORD sample_size, DWORD *flags, bool *preserve_timestamps) { struct wg_sample *wg_sample; IMFMediaBuffer *buffer; @@ -368,6 +368,8 @@ HRESULT wg_transform_read_mf(wg_transform_t transform, IMFSample *sample, IMFSample_SetUINT32(sample, &MFSampleExtension_CleanPoint, 1); if (wg_sample->flags & WG_SAMPLE_FLAG_DISCONTINUITY) IMFSample_SetUINT32(sample, &MFSampleExtension_Discontinuity, 1); + if (preserve_timestamps) + *preserve_timestamps = !!(wg_sample->flags & WG_SAMPLE_FLAG_PRESERVE_TIMESTAMPS); if (SUCCEEDED(hr = IMFSample_ConvertToContiguousBuffer(sample, &buffer))) { diff --git a/dlls/winegstreamer/wg_transform.c b/dlls/winegstreamer/wg_transform.c index 5c1f2fd62bc..01f8a6716da 100644 --- a/dlls/winegstreamer/wg_transform.c +++ b/dlls/winegstreamer/wg_transform.c @@ -820,6 +820,7 @@ NTSTATUS wg_transform_push_data(void *args) struct wg_transform_push_data_params *params = args; struct wg_transform *transform = get_transform(params->transform); struct wg_sample *sample = params->sample; + GstCaps *transform_timestamp; const gchar *input_mime; GstVideoInfo video_info; GstBuffer *buffer; @@ -868,6 +869,14 @@ NTSTATUS wg_transform_push_data(void *args) GST_BUFFER_FLAG_SET(buffer, GST_BUFFER_FLAG_DELTA_UNIT); if (sample->flags & WG_SAMPLE_FLAG_DISCONTINUITY) GST_BUFFER_FLAG_SET(buffer, GST_BUFFER_FLAG_DISCONT); + + if (transform->attrs.preserve_timestamps && (sample->flags & WG_SAMPLE_FLAG_HAS_PTS) + && (transform_timestamp = gst_caps_new_empty_simple("timestamp/x-wg-transform"))) + { + gst_buffer_add_reference_timestamp_meta(buffer, transform_timestamp, GST_BUFFER_PTS(buffer), GST_BUFFER_DURATION(buffer)); + gst_caps_unref(transform_timestamp); + } + gst_atomic_queue_push(transform->input_queue, buffer); params->result = S_OK; @@ -944,22 +953,43 @@ static NTSTATUS copy_buffer(GstBuffer *buffer, struct wg_sample *sample, gsize * static void set_sample_flags_from_buffer(struct wg_sample *sample, GstBuffer *buffer, gsize total_size) { - if (GST_BUFFER_PTS_IS_VALID(buffer)) + GstReferenceTimestampMeta *timestamps; + GstCaps *transform_timestamp; + + transform_timestamp = gst_caps_new_empty_simple("timestamp/x-wg-transform"); + timestamps = gst_buffer_get_reference_timestamp_meta(buffer, transform_timestamp); + gst_caps_unref(transform_timestamp); + + if (timestamps) { - sample->flags |= WG_SAMPLE_FLAG_HAS_PTS; - sample->pts = GST_BUFFER_PTS(buffer) / 100; + /* GStreamer can overwrite our timestamps, so we use the wg-transform timestamps instead */ + sample->flags |= WG_SAMPLE_FLAG_HAS_PTS | WG_SAMPLE_FLAG_PRESERVE_TIMESTAMPS; + sample->pts = timestamps->timestamp / 100; + if (timestamps->duration != GST_CLOCK_TIME_NONE) + { + sample->flags |= WG_SAMPLE_FLAG_HAS_DURATION; + sample->duration = timestamps->duration / 100; + } } - if (GST_BUFFER_DURATION_IS_VALID(buffer)) + else { - GstClockTime duration = GST_BUFFER_DURATION(buffer) / 100; - - duration = (duration * sample->size) / total_size; - GST_BUFFER_DURATION(buffer) -= duration * 100; if (GST_BUFFER_PTS_IS_VALID(buffer)) - GST_BUFFER_PTS(buffer) += duration * 100; + { + sample->flags |= WG_SAMPLE_FLAG_HAS_PTS; + sample->pts = GST_BUFFER_PTS(buffer) / 100; + } + if (GST_BUFFER_DURATION_IS_VALID(buffer)) + { + GstClockTime duration = GST_BUFFER_DURATION(buffer) / 100; + + duration = (duration * sample->size) / total_size; + GST_BUFFER_DURATION(buffer) -= duration * 100; + if (GST_BUFFER_PTS_IS_VALID(buffer)) + GST_BUFFER_PTS(buffer) += duration * 100; - sample->flags |= WG_SAMPLE_FLAG_HAS_DURATION; - sample->duration = duration; + sample->flags |= WG_SAMPLE_FLAG_HAS_DURATION; + sample->duration = duration; + } } if (!GST_BUFFER_FLAG_IS_SET(buffer, GST_BUFFER_FLAG_DELTA_UNIT)) sample->flags |= WG_SAMPLE_FLAG_SYNC_POINT; diff --git a/dlls/winegstreamer/wma_decoder.c b/dlls/winegstreamer/wma_decoder.c index 09b2ee647b3..312dec2034e 100644 --- a/dlls/winegstreamer/wma_decoder.c +++ b/dlls/winegstreamer/wma_decoder.c @@ -557,7 +557,7 @@ static HRESULT WINAPI transform_ProcessOutput(IMFTransform *iface, DWORD flags, return hr; if (SUCCEEDED(hr = wg_transform_read_mf(decoder->wg_transform, samples->pSample, - info.cbSize, &samples->dwStatus))) + info.cbSize, &samples->dwStatus, NULL))) wg_sample_queue_flush(decoder->wg_sample_queue, false); return hr; -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/7623
From: Brendan McGrath <bmcgrath(a)codeweavers.com> --- dlls/mf/tests/transform.c | 4 ---- dlls/winegstreamer/video_decoder.c | 2 +- 2 files changed, 1 insertion(+), 5 deletions(-) diff --git a/dlls/mf/tests/transform.c b/dlls/mf/tests/transform.c index d450c4aecec..316680cf01b 100644 --- a/dlls/mf/tests/transform.c +++ b/dlls/mf/tests/transform.c @@ -5285,11 +5285,9 @@ static void test_h264_decoder_timestamps(void) { hr = IMFSample_GetSampleTime(output_sample, &time); ok(hr == S_OK, "Got %#lx\n", hr); - todo_wine_if(i) ok(time == exp_nofps[i].time, "got time %I64d, expected %I64d\n", time, exp_nofps[i].time); hr = IMFSample_GetSampleDuration(output_sample, &duration); ok(hr == S_OK, "Got %#lx\n", hr); - todo_wine ok(duration == exp_nofps[i].duration, "got duration %I64d, expected %I64d\n", duration, exp_nofps[i].duration); ret = IMFSample_Release(output_sample); ok(ret == 0, "Release returned %lu\n", ret); @@ -5335,11 +5333,9 @@ static void test_h264_decoder_timestamps(void) { hr = IMFSample_GetSampleTime(output_sample, &time); ok(hr == S_OK, "Got %#lx\n", hr); - todo_wine_if(i) ok(time == exp_24fps[i].time, "got time %I64d, expected %I64d\n", time, exp_24fps[i].time); hr = IMFSample_GetSampleDuration(output_sample, &duration); ok(hr == S_OK, "Got %#lx\n", hr); - todo_wine ok(duration == exp_24fps[i].duration, "got duration %I64d, expected %I64d\n", duration, exp_24fps[i].duration); ret = IMFSample_Release(output_sample); ok(ret == 0, "Release returned %lu\n", ret); diff --git a/dlls/winegstreamer/video_decoder.c b/dlls/winegstreamer/video_decoder.c index 47b2eb0371e..57a2ebf1572 100644 --- a/dlls/winegstreamer/video_decoder.c +++ b/dlls/winegstreamer/video_decoder.c @@ -999,7 +999,7 @@ static HRESULT WINAPI transform_ProcessOutput(IMFTransform *iface, DWORD flags, if (FAILED(IMFMediaType_GetUINT64(decoder->input_type, &MF_MT_FRAME_RATE, &frame_rate))) frame_rate = (UINT64)30000 << 32 | 1001; - duration = (UINT64)10000000 * (UINT32)frame_rate / (frame_rate >> 32); + duration = MulDiv(10000000, (UINT32)frame_rate, frame_rate >> 32); if (FAILED(IMFSample_SetSampleTime(sample, decoder->sample_time))) WARN("Failed to set sample time\n"); if (FAILED(IMFSample_SetSampleDuration(sample, duration))) -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/7623
From: Brendan McGrath <bmcgrath(a)codeweavers.com> --- dlls/mf/tests/transform.c | 4 ---- dlls/winegstreamer/video_decoder.c | 9 +++++++-- 2 files changed, 7 insertions(+), 6 deletions(-) diff --git a/dlls/mf/tests/transform.c b/dlls/mf/tests/transform.c index 316680cf01b..b1d72ddddc3 100644 --- a/dlls/mf/tests/transform.c +++ b/dlls/mf/tests/transform.c @@ -7283,11 +7283,9 @@ static void test_wmv_decoder_timestamps(void) { hr = IMFSample_GetSampleTime(output_sample, &time); ok(hr == S_OK, "Got %#lx\n", hr); - todo_wine_if(i) ok(time == exp_nofps[i].time, "got time %I64d, expected %I64d\n", time, exp_nofps[i].time); hr = IMFSample_GetSampleDuration(output_sample, &duration); ok(hr == S_OK, "Got %#lx\n", hr); - todo_wine ok(duration == exp_nofps[i].duration, "got duration %I64d, expected %I64d\n", duration, exp_nofps[i].duration); ret = IMFSample_Release(output_sample); ok(ret == 0, "Release returned %lu\n", ret); @@ -7341,11 +7339,9 @@ static void test_wmv_decoder_timestamps(void) { hr = IMFSample_GetSampleTime(output_sample, &time); ok(hr == S_OK, "Got %#lx\n", hr); - todo_wine_if(i) ok(time == exp_24fps[i].time, "got time %I64d, expected %I64d\n", time, exp_24fps[i].time); hr = IMFSample_GetSampleDuration(output_sample, &duration); ok(hr == S_OK, "Got %#lx\n", hr); - todo_wine ok(duration == exp_24fps[i].duration, "got duration %I64d, expected %I64d\n", duration, exp_24fps[i].duration); ret = IMFSample_Release(output_sample); ok(ret == 0, "Release returned %lu\n", ret); diff --git a/dlls/winegstreamer/video_decoder.c b/dlls/winegstreamer/video_decoder.c index 57a2ebf1572..a65f1781f96 100644 --- a/dlls/winegstreamer/video_decoder.c +++ b/dlls/winegstreamer/video_decoder.c @@ -994,12 +994,17 @@ static HRESULT WINAPI transform_ProcessOutput(IMFTransform *iface, DWORD flags, { wg_sample_queue_flush(decoder->wg_sample_queue, false); - if (!preserve_timestamps) + if (decoder->IMediaObject_iface.lpVtbl) + duration = 0; /* WMV decoder doesn't output any timestamp or duration */ + else { if (FAILED(IMFMediaType_GetUINT64(decoder->input_type, &MF_MT_FRAME_RATE, &frame_rate))) frame_rate = (UINT64)30000 << 32 | 1001; - duration = MulDiv(10000000, (UINT32)frame_rate, frame_rate >> 32); + } + + if (!preserve_timestamps) + { if (FAILED(IMFSample_SetSampleTime(sample, decoder->sample_time))) WARN("Failed to set sample time\n"); if (FAILED(IMFSample_SetSampleDuration(sample, duration))) -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/7623
From: Brendan McGrath <bmcgrath(a)codeweavers.com> --- dlls/mf/tests/transform.c | 1 - dlls/winegstreamer/unixlib.h | 2 +- dlls/winegstreamer/wg_transform.c | 19 ++++++++++++++++++- 3 files changed, 19 insertions(+), 3 deletions(-) diff --git a/dlls/mf/tests/transform.c b/dlls/mf/tests/transform.c index b1d72ddddc3..4578dbac591 100644 --- a/dlls/mf/tests/transform.c +++ b/dlls/mf/tests/transform.c @@ -5446,7 +5446,6 @@ static void test_h264_decoder_timestamps(void) { hr = IMFSample_GetSampleTime(output_sample, &time); ok(hr == S_OK, "Got %#lx\n", hr); - todo_wine_if(exp_sample_neg_ts[i].time < 0) ok(time == exp_sample_neg_ts[i].time, "got time %I64d, expected %I64d\n", time, exp_sample_neg_ts[i].time); hr = IMFSample_GetSampleDuration(output_sample, &duration); ok(hr == S_OK, "Got %#lx\n", hr); diff --git a/dlls/winegstreamer/unixlib.h b/dlls/winegstreamer/unixlib.h index d1a414b6396..83e38849fa9 100644 --- a/dlls/winegstreamer/unixlib.h +++ b/dlls/winegstreamer/unixlib.h @@ -185,7 +185,7 @@ enum wg_sample_flag struct wg_sample { /* timestamp and duration are in 100-nanosecond units. */ - UINT64 pts; + INT64 pts; UINT64 duration; LONG refcount; /* unix refcount */ UINT32 flags; diff --git a/dlls/winegstreamer/wg_transform.c b/dlls/winegstreamer/wg_transform.c index 01f8a6716da..1e666bd7ff0 100644 --- a/dlls/winegstreamer/wg_transform.c +++ b/dlls/winegstreamer/wg_transform.c @@ -96,6 +96,7 @@ struct wg_transform GstCaps *input_caps; bool draining; + INT64 ts_offset; }; static struct wg_transform *get_transform(wg_transform_t trans) @@ -862,7 +863,19 @@ NTSTATUS wg_transform_push_data(void *args) } if (sample->flags & WG_SAMPLE_FLAG_HAS_PTS) - GST_BUFFER_PTS(buffer) = sample->pts * 100; + { + if (sample->pts < transform->ts_offset) + { + if (transform->ts_offset) + GST_FIXME("ts_offset is already set to %"GST_TIME_FORMAT", overwriting", + GST_TIME_ARGS(-transform->ts_offset)); + + GST_TRACE("Setting ts_offset to %"GST_TIME_FORMAT, GST_TIME_ARGS(-sample->pts)); + transform->ts_offset = sample->pts; + } + + GST_BUFFER_PTS(buffer) = (sample->pts - transform->ts_offset) * 100; + } if (sample->flags & WG_SAMPLE_FLAG_HAS_DURATION) GST_BUFFER_DURATION(buffer) = sample->duration * 100; if (!(sample->flags & WG_SAMPLE_FLAG_SYNC_POINT)) @@ -1196,6 +1209,10 @@ NTSTATUS wg_transform_read_data(void *args) else status = read_transform_output(sample, output_buffer); + if ((sample->flags & (WG_SAMPLE_FLAG_PRESERVE_TIMESTAMPS | WG_SAMPLE_FLAG_HAS_PTS)) == + (WG_SAMPLE_FLAG_PRESERVE_TIMESTAMPS | WG_SAMPLE_FLAG_HAS_PTS)) + sample->pts += transform->ts_offset; + if (status) { wg_allocator_release_sample(transform->allocator, sample, false); -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/7623
On Fri May 16 00:18:05 2025 +0000, Brendan McGrath wrote:
changed this line in [version 10 of the diff](/wine/wine/-/merge_requests/7623/diffs?diff_id=178254&start_sha=fc34e7fba1b6a5b2619dccf42e7c8024653038a6#c08e9e1a7300fe731e927110e87680757465897f_25_25) I just adopted your branch, so `bool *` it is
-- https://gitlab.winehq.org/wine/wine/-/merge_requests/7623#note_103565
On Fri May 16 00:18:05 2025 +0000, Brendan McGrath wrote:
changed this line in [version 10 of the diff](/wine/wine/-/merge_requests/7623/diffs?diff_id=178254&start_sha=fc34e7fba1b6a5b2619dccf42e7c8024653038a6#bd62eaa50df6790a4fd6a511972092cad3bb6fa1_1013_1006) Fair. I'll keep that in mind moving forward.
-- https://gitlab.winehq.org/wine/wine/-/merge_requests/7623#note_103566
On Fri May 16 00:31:25 2025 +0000, Rémi Bernon wrote:
What about something like https://gitlab.winehq.org/rbernon/wine/-/commits/mr/7623? I've split the changes to make individual fixups more obvious and made other tweaks. If you think it's okay please update the MR with that branch, I would approve then. That branch looks good, so I have adopted it. There was one line I didn't fully understand though: https://gitlab.winehq.org/wine/wine/-/merge_requests/7623/diffs#bd62eaa50df6...
``` if (decoder->IMediaObject_iface.lpVtbl) ``` Is that something that will hold true if we add more codecs (or change existing ones)? -- https://gitlab.winehq.org/wine/wine/-/merge_requests/7623#note_103568
Is that something that will hold true if we add more codecs (or change existing ones)?
Not necessarily but it's been used to distinguish the WMV decoder from others already in some other part of the code. There's only tests for the H264 and the WMV decoder but it seems to me that not generating timestamps/duration is probably the special case here. I'm even surprised we need to strip them, I suspect it is specific to the IMFTransform interface of the WMV decoder as otherwise I don't see how things could work without timestamps otherwise. -- https://gitlab.winehq.org/wine/wine/-/merge_requests/7623#note_103591
This merge request was approved by Rémi Bernon. -- https://gitlab.winehq.org/wine/wine/-/merge_requests/7623
participants (3)
-
Brendan McGrath -
Brendan McGrath (＠redmcg)
-
Rémi Bernon