[PATCH 0/4] MR10421: winegstreamer: Improve quartz MPEG-1 video seeking.
This MR improves quartz video seeking particularly for MPEG-1 video. The quartz decoder now flushes the wg_transform buffers on seek. Furthermore, the GStreamer plugin, avdec_mpeg2video, now receives additional information equivalent to an MPEG-1 sequence header, allowing seeking even when such headers are not regularly present. If you're looking for a test video, I recommend the Kirby video available at [1]. It contains only a single sequence header, which makes it suitable for testing. [1] https://www.nintendo.co.jp/ngc/gkyj/movies/htp.mpg -- https://gitlab.winehq.org/wine/wine/-/merge_requests/10421
From: Akihiro Sagawa <sagawa.aki@gmail.com> --- dlls/winegstreamer/quartz_transform.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/dlls/winegstreamer/quartz_transform.c b/dlls/winegstreamer/quartz_transform.c index 5b4b9e77cf9..63451f5b45c 100644 --- a/dlls/winegstreamer/quartz_transform.c +++ b/dlls/winegstreamer/quartz_transform.c @@ -413,11 +413,29 @@ static HRESULT WINAPI transform_sink_receive(struct strmbase_sink *pin, IMediaSa return S_OK; } +static HRESULT transform_sink_begin_flush(struct strmbase_sink *pin) +{ + struct transform *filter = impl_from_strmbase_filter(pin->pin.filter); + HRESULT hr; + + TRACE("(%p)\n", pin); + + hr = wg_transform_flush(filter->transform); + + wg_sample_queue_flush(filter->sample_queue, true); + + if (filter->source.pin.peer) + IPin_BeginFlush(filter->source.pin.peer); + + return hr; +} + static const struct strmbase_sink_ops sink_ops = { .base.pin_query_accept = transform_sink_query_accept, .base.pin_query_interface = transform_sink_query_interface, .pfnReceive = transform_sink_receive, + .sink_begin_flush = transform_sink_begin_flush, }; static HRESULT transform_source_query_accept(struct strmbase_pin *pin, const AM_MEDIA_TYPE *mt) -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/10421
From: Akihiro Sagawa <sagawa.aki@gmail.com> --- dlls/quartz/tests/mpegsplit.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/dlls/quartz/tests/mpegsplit.c b/dlls/quartz/tests/mpegsplit.c index 3d4ba8b7959..39d1269a2f7 100644 --- a/dlls/quartz/tests/mpegsplit.c +++ b/dlls/quartz/tests/mpegsplit.c @@ -2027,10 +2027,12 @@ static void test_streamselect(IAMStreamSelect *sel) static void test_video_file(void) { + static const unsigned char SEQ_START_CODE[] = {0, 0, 1, 0xb3}; const WCHAR *filename = load_resource(L"test.mpg"); IBaseFilter *filter = create_mpeg_splitter(); struct testfilter testsink_video; struct testfilter testsink_audio; + const MPEG1VIDEOINFO *format; IPin *source_video = NULL; IPin *source_audio = NULL; IMediaControl *control; @@ -2075,6 +2077,13 @@ static void test_video_file(void) ok(IsEqualGUID(&testsink_video.sink.pin.mt.subtype, &MEDIASUBTYPE_MPEG1Payload), "Media types didn't match.\n"); ok(IsEqualGUID(&testsink_video.sink.pin.mt.formattype, &FORMAT_MPEGVideo), "Media types didn't match.\n"); + format = (const MPEG1VIDEOINFO *)testsink_video.sink.pin.mt.pbFormat; + ret = offsetof(MPEG1VIDEOINFO, bSequenceHeader[0]) + format->cbSequenceHeader; + todo_wine ok(testsink_video.sink.pin.mt.cbFormat == ret, "cbFormat didn't match, got %lu.\n", testsink_video.sink.pin.mt.cbFormat); + todo_wine ok(format->dwStartTimeCode == 4096, "dwStartTimeCode didn't match, got %lu.\n", format->dwStartTimeCode); + todo_wine ok(format->cbSequenceHeader == 12, "cbSequenceHeader didn't match, got %lu.\n", format->cbSequenceHeader); + todo_wine ok(!memcmp(format->bSequenceHeader, SEQ_START_CODE, sizeof(SEQ_START_CODE)), "Sequence header didn't match.\n"); + ok(IsEqualGUID(&testsink_audio.sink.pin.mt.majortype, &MEDIATYPE_Audio), "Media types didn't match.\n"); ok(IsEqualGUID(&testsink_audio.sink.pin.mt.subtype, &MEDIASUBTYPE_MPEG1AudioPayload), "Media types didn't match.\n"); ok(IsEqualGUID(&testsink_audio.sink.pin.mt.formattype, &FORMAT_WaveFormatEx), "Media types didn't match.\n"); -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/10421
From: Akihiro Sagawa <sagawa.aki@gmail.com> Unlike DVD videos, MPEG-1 video does not require periodic sequence headers. With this change, the decoder now receives additional information equivalent to a sequence header, allowing seeking even when such headers are not regularly present. --- dlls/quartz/tests/mpegsplit.c | 6 ++--- dlls/winegstreamer/main.c | 2 +- dlls/winegstreamer/quartz_parser.c | 41 +++++++++++++++++++++++++++--- dlls/winegstreamer/unixlib.h | 4 +-- dlls/winegstreamer/wg_format.c | 34 +++++++++++++++++++++++++ 5 files changed, 77 insertions(+), 10 deletions(-) diff --git a/dlls/quartz/tests/mpegsplit.c b/dlls/quartz/tests/mpegsplit.c index 39d1269a2f7..1d62f0000ee 100644 --- a/dlls/quartz/tests/mpegsplit.c +++ b/dlls/quartz/tests/mpegsplit.c @@ -2079,10 +2079,10 @@ static void test_video_file(void) format = (const MPEG1VIDEOINFO *)testsink_video.sink.pin.mt.pbFormat; ret = offsetof(MPEG1VIDEOINFO, bSequenceHeader[0]) + format->cbSequenceHeader; - todo_wine ok(testsink_video.sink.pin.mt.cbFormat == ret, "cbFormat didn't match, got %lu.\n", testsink_video.sink.pin.mt.cbFormat); + ok(testsink_video.sink.pin.mt.cbFormat == ret, "cbFormat didn't match, got %lu.\n", testsink_video.sink.pin.mt.cbFormat); todo_wine ok(format->dwStartTimeCode == 4096, "dwStartTimeCode didn't match, got %lu.\n", format->dwStartTimeCode); - todo_wine ok(format->cbSequenceHeader == 12, "cbSequenceHeader didn't match, got %lu.\n", format->cbSequenceHeader); - todo_wine ok(!memcmp(format->bSequenceHeader, SEQ_START_CODE, sizeof(SEQ_START_CODE)), "Sequence header didn't match.\n"); + ok(format->cbSequenceHeader == 12, "cbSequenceHeader didn't match, got %lu.\n", format->cbSequenceHeader); + ok(!memcmp(format->bSequenceHeader, SEQ_START_CODE, sizeof(SEQ_START_CODE)), "Sequence header didn't match.\n"); ok(IsEqualGUID(&testsink_audio.sink.pin.mt.majortype, &MEDIATYPE_Audio), "Media types didn't match.\n"); ok(IsEqualGUID(&testsink_audio.sink.pin.mt.subtype, &MEDIASUBTYPE_MPEG1AudioPayload), "Media types didn't match.\n"); diff --git a/dlls/winegstreamer/main.c b/dlls/winegstreamer/main.c index 45c36e07ca2..c7893f71026 100644 --- a/dlls/winegstreamer/main.c +++ b/dlls/winegstreamer/main.c @@ -92,7 +92,7 @@ static HRESULT video_format_from_media_type(IMFMediaType *media_type, MFVIDEOFOR memset(mpeg, 0, mpeg_size); mpeg->hdr = **format; - IMFMediaType_GetBlob(media_type, &MF_MT_MPEG_SEQUENCE_HEADER, mpeg->sequence_header, len, NULL); + IMFMediaType_GetBlob(media_type, &MF_MT_MPEG_SEQUENCE_HEADER, mpeg->sequence_header, len, &mpeg->sequence_header_count); IMFMediaType_GetUINT32(media_type, &MF_MT_MPEG_START_TIME_CODE, (UINT32 *)&mpeg->start_time_code); IMFMediaType_GetUINT32(media_type, &MF_MT_MPEG2_PROFILE, &mpeg->profile); IMFMediaType_GetUINT32(media_type, &MF_MT_MPEG2_LEVEL, &mpeg->level); diff --git a/dlls/winegstreamer/quartz_parser.c b/dlls/winegstreamer/quartz_parser.c index f09e8b9382a..712a10be4b9 100644 --- a/dlls/winegstreamer/quartz_parser.c +++ b/dlls/winegstreamer/quartz_parser.c @@ -745,8 +745,10 @@ static bool amt_from_wg_format_video_mpeg1(AM_MEDIA_TYPE *mt, const struct wg_fo { MPEG1VIDEOINFO *video_format; uint32_t frame_time; + DWORD size; - if (!(video_format = CoTaskMemAlloc(sizeof(*video_format)))) + size = offsetof(MPEG1VIDEOINFO, bSequenceHeader) + format->u.video.codec_data_len; + if (!(video_format = CoTaskMemAlloc(size))) return false; mt->majortype = MEDIATYPE_Video; @@ -754,10 +756,10 @@ static bool amt_from_wg_format_video_mpeg1(AM_MEDIA_TYPE *mt, const struct wg_fo mt->bTemporalCompression = TRUE; mt->lSampleSize = 1; mt->formattype = FORMAT_MPEGVideo; - mt->cbFormat = sizeof(MPEG1VIDEOINFO); + mt->cbFormat = size; mt->pbFormat = (BYTE *)video_format; - memset(video_format, 0, sizeof(*video_format)); + memset(video_format, 0, size); if ((frame_time = MulDiv(10000000, format->u.video.fps_d, format->u.video.fps_n)) != -1) video_format->hdr.AvgTimePerFrame = frame_time; video_format->hdr.bmiHeader.biSize = sizeof(BITMAPINFOHEADER); @@ -768,6 +770,30 @@ static bool amt_from_wg_format_video_mpeg1(AM_MEDIA_TYPE *mt, const struct wg_fo video_format->hdr.bmiHeader.biCompression = mt->subtype.Data1; video_format->hdr.bmiHeader.biSizeImage = wg_format_get_max_size(format); + /* Ignore additional start code units after the sequence header. */ + size = 12; /* minimum sequence header length */ + if (format->u.video.codec_data_len >= size + && format->u.video.codec_data[0] == 0x00 + && format->u.video.codec_data[1] == 0x00 + && format->u.video.codec_data[2] == 0x01 + && format->u.video.codec_data[3] == 0xb3) + { + /* has intra quantiser matrix? */ + if (format->u.video.codec_data[size-1] & 2) + size += 64; + + /* has non-intra quantiser matrix? */ + if (format->u.video.codec_data_len >= size + && (format->u.video.codec_data[size-1] & 1)) + size += 64; + + if (format->u.video.codec_data_len >= size) + { + video_format->cbSequenceHeader = size; + memcpy(video_format->bSequenceHeader, format->u.video.codec_data, size); + } + } + return true; } @@ -1136,7 +1162,7 @@ static bool amt_to_wg_format_video_mpeg1(const AM_MEDIA_TYPE *mt, struct wg_form FIXME("Unknown format type %s.\n", debugstr_guid(&mt->formattype)); return false; } - if (mt->cbFormat < sizeof(VIDEOINFOHEADER) || !mt->pbFormat) + if (!mt->pbFormat || mt->cbFormat < offsetof(MPEG1VIDEOINFO, bSequenceHeader) + video_format->cbSequenceHeader) { ERR("Unexpected format size %lu.\n", mt->cbFormat); return false; @@ -1148,6 +1174,13 @@ static bool amt_to_wg_format_video_mpeg1(const AM_MEDIA_TYPE *mt, struct wg_form format->u.video.fps_n = 10000000; format->u.video.fps_d = video_format->hdr.AvgTimePerFrame; + format->u.video.codec_data_len = video_format->cbSequenceHeader; + if (format->u.video.codec_data_len > sizeof(format->u.video.codec_data)) + { + ERR("Too big codec_data value (%u).\n", format->u.video.codec_data_len); + format->u.video.codec_data_len = 0; + } + memcpy(format->u.video.codec_data, video_format->bSequenceHeader, format->u.video.codec_data_len); return true; } diff --git a/dlls/winegstreamer/unixlib.h b/dlls/winegstreamer/unixlib.h index 5d11934fac2..f545df7ad0b 100644 --- a/dlls/winegstreamer/unixlib.h +++ b/dlls/winegstreamer/unixlib.h @@ -154,7 +154,7 @@ struct wg_format * H264: width, height, fps_n, fps_d, profile, level, codec_data_len, codec_data. * WMV: width, height, fps_n, fps_d, codec_data_len, codec_data. * INDEO: width, height, fps_n, fps_d, version. - * MPEG1: width, height, fps_n, fps_d. */ + * MPEG1: width, height, fps_n, fps_d, codec_data_len, codec_data. */ struct { wg_video_format format; @@ -168,7 +168,7 @@ struct wg_format uint32_t level; uint32_t version; uint32_t codec_data_len; - unsigned char codec_data[64]; + unsigned char codec_data[140]; /* =MAX_SIZE_MPEG1_SEQUENCE_INFO */ } video; } u; }; diff --git a/dlls/winegstreamer/wg_format.c b/dlls/winegstreamer/wg_format.c index bed7ffb2964..fce3d27a0d3 100644 --- a/dlls/winegstreamer/wg_format.c +++ b/dlls/winegstreamer/wg_format.c @@ -470,6 +470,10 @@ static void wg_format_from_caps_video_mpeg1(struct wg_format *format, const GstC { const GstStructure *structure = gst_caps_get_structure(caps, 0); gint width, height, fps_n, fps_d; + const GValue *codec_data_value; + GstBuffer *codec_data; + GstMapInfo map; + guint size; if (!gst_structure_get_int(structure, "width", &width)) { @@ -481,6 +485,12 @@ static void wg_format_from_caps_video_mpeg1(struct wg_format *format, const GstC GST_WARNING("Missing \"height\" value in %" GST_PTR_FORMAT ".", caps); return; } + if (!(codec_data_value = gst_structure_get_value(structure, "codec_data")) + || !(codec_data = gst_value_get_buffer(codec_data_value))) + { + GST_WARNING("Missing \"codec_data\" value in %" GST_PTR_FORMAT ".", caps); + return; + } if (!gst_structure_get_fraction(structure, "framerate", &fps_n, &fps_d)) { fps_n = 0; @@ -492,6 +502,15 @@ static void wg_format_from_caps_video_mpeg1(struct wg_format *format, const GstC format->u.video.height = height; format->u.video.fps_n = fps_n; format->u.video.fps_d = fps_d; + + gst_buffer_map(codec_data, &map, GST_MAP_READ); + + assert(sizeof(format->u.video.codec_data) >= MAX_SIZE_MPEG1_SEQUENCE_INFO); + size = min(map.size, MAX_SIZE_MPEG1_SEQUENCE_INFO); + format->u.video.codec_data_len = size; + memcpy(format->u.video.codec_data, map.data, size); + + gst_buffer_unmap(codec_data, &map); } static const struct @@ -1077,6 +1096,7 @@ static GstCaps *wg_format_to_caps_video_indeo(const struct wg_format *format) static GstCaps *wg_format_to_caps_video_mpeg1(const struct wg_format *format) { + GstBuffer *buffer; GstCaps *caps; if (!(caps = gst_caps_new_empty_simple("video/mpeg"))) @@ -1091,6 +1111,20 @@ static GstCaps *wg_format_to_caps_video_mpeg1(const struct wg_format *format) gst_caps_set_simple(caps, "height", G_TYPE_INT, format->u.video.height, NULL); if (format->u.video.fps_d || format->u.video.fps_n) gst_caps_set_simple(caps, "framerate", GST_TYPE_FRACTION, format->u.video.fps_n, format->u.video.fps_d, NULL); + + if (format->u.video.codec_data_len) + { + if (!(buffer = gst_buffer_new_and_alloc(format->u.video.codec_data_len))) + { + gst_caps_unref(caps); + return NULL; + } + + gst_buffer_fill(buffer, 0, format->u.video.codec_data, format->u.video.codec_data_len); + gst_caps_set_simple(caps, "codec_data", GST_TYPE_BUFFER, buffer, NULL); + gst_buffer_unref(buffer); + } + return caps; } -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/10421
From: Akihiro Sagawa <sagawa.aki@gmail.com> Otherwise, it sticks after seek. --- dlls/winegstreamer/quartz_parser.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dlls/winegstreamer/quartz_parser.c b/dlls/winegstreamer/quartz_parser.c index 712a10be4b9..38971f2fcb9 100644 --- a/dlls/winegstreamer/quartz_parser.c +++ b/dlls/winegstreamer/quartz_parser.c @@ -2303,7 +2303,7 @@ static HRESULT WINAPI GSTOutPin_DecideBufferSize(struct strmbase_source *iface, { MPEG1VIDEOINFO *format = (MPEG1VIDEOINFO *)pin->pin.pin.mt.pbFormat; buffer_size = format->hdr.bmiHeader.biSizeImage; - buffer_count = 8; + buffer_count = 30; } else if (IsEqualGUID(&pin->pin.pin.mt.formattype, &FORMAT_WaveFormatEx) && (IsEqualGUID(&pin->pin.pin.mt.subtype, &MEDIASUBTYPE_PCM) -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/10421
This actually fix https://bugs.winehq.org/show_bug.cgi?id=55933 but not perfect fix, it will stop a bit when video play to last frame, before start again. -- https://gitlab.winehq.org/wine/wine/-/merge_requests/10421#note_133257
This looks good, but 4/4 gives me pause. It's not wrong, and actually testing shows that native seems to go higher, allocating 100 samples. But why would it stick after seeking with a more limited pool? -- https://gitlab.winehq.org/wine/wine/-/merge_requests/10421#note_133583
This merge request was approved by Elizabeth Figura. -- https://gitlab.winehq.org/wine/wine/-/merge_requests/10421
Looks like there are a couple todo_wine in quartz:mpegaudio that need to be removed. -- https://gitlab.winehq.org/wine/wine/-/merge_requests/10421#note_133593
participants (3)
-
Akihiro Sagawa -
Akihiro Sagawa (＠sgwaki)
-
Chunhao Hung (＠otakuxtom)
-
Elizabeth Figura (＠zfigura)