[PATCH v4 0/12] MR5159: Draft: Overwatch highlight/POTG encoding (part 1)
Part 1 of a set of changes for making the recording of highlights and "Play of the Game"s to mp4 in Overwatch work. This MR contains most of the required IMFMediaSink changes. -- v4: winegstreamer: Avoid an assert in Gstreamer code. mfplat: Return NULL for *stream_sink on error in IMFMediaSink::AddStreamSink. mfplat: Accept any media types without a MF_MT_USER_DATA attribute in MFCreateWaveFormatExFromMFMediaType. mf/tests: Add test for MEStreamSinkStarted and MEStreamSinkRequestSample events. winegstreamer: Add support for ADTS container format (.aac files). winegstreamer: Request new samples after starting media sink and after processing a sample. mf/tests: Test media sink creation with a media type without MF_MT_USER_DATA attribute. winegstreamer: Generate AAC codec data if not provided in a media type. winegstreamer: Implement IMFStreamSink::IMFMediaTypeHandler.GetMajorType. winegstreamer: Copy media type instead of returning a reference in IMFStreamSink::IMFMediaTypeHandler.GetCurrentMediaType. winegstreamer: Implement IMFMediaSink::GetPresentationClock. winegstreamer: Implement IMFMediaSink::SetPresentationClock. https://gitlab.winehq.org/wine/wine/-/merge_requests/5159
From: Torge Matthies <tmatthies(a)codeweavers.com> --- dlls/mf/tests/mf.c | 3 --- dlls/winegstreamer/media_sink.c | 41 +++++++++++++++++++++++++++++++-- 2 files changed, 39 insertions(+), 5 deletions(-) diff --git a/dlls/mf/tests/mf.c b/dlls/mf/tests/mf.c index fece60d37a7..2f288e854d7 100644 --- a/dlls/mf/tests/mf.c +++ b/dlls/mf/tests/mf.c @@ -7806,11 +7806,8 @@ static void test_mpeg4_media_sink(void) hr = MFCreatePresentationClock(&clock); ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); hr = IMFMediaSink_SetPresentationClock(sink, NULL); - todo_wine ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); - todo_wine hr = IMFMediaSink_SetPresentationClock(sink, clock); - todo_wine ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); IMFPresentationClock_Release(clock); diff --git a/dlls/winegstreamer/media_sink.c b/dlls/winegstreamer/media_sink.c index 5ee2c44dc70..0d76c6f30b9 100644 --- a/dlls/winegstreamer/media_sink.c +++ b/dlls/winegstreamer/media_sink.c @@ -91,6 +91,7 @@ struct media_sink IMFByteStream *bytestream; IMFMediaEventQueue *event_queue; + IMFPresentationClock *clock; struct list stream_sinks; @@ -877,9 +878,40 @@ static HRESULT WINAPI media_sink_GetStreamSinkById(IMFFinalizableMediaSink *ifac static HRESULT WINAPI media_sink_SetPresentationClock(IMFFinalizableMediaSink *iface, IMFPresentationClock *clock) { - FIXME("iface %p, clock %p stub!\n", iface, clock); + struct media_sink *media_sink = impl_from_IMFFinalizableMediaSink(iface); + HRESULT hr = S_OK; - return E_NOTIMPL; + TRACE("iface %p, clock %p\n", iface, clock); + + EnterCriticalSection(&media_sink->cs); + + if (media_sink->state == STATE_SHUTDOWN) + { + hr = MF_E_SHUTDOWN; + goto done; + } + + if (media_sink->clock) + { + hr = IMFPresentationClock_RemoveClockStateSink(media_sink->clock, &media_sink->IMFClockStateSink_iface); + if (FAILED(hr)) + goto done; + IMFPresentationClock_Release(media_sink->clock); + media_sink->clock = NULL; + } + + if (clock) + { + hr = IMFPresentationClock_AddClockStateSink(clock, &media_sink->IMFClockStateSink_iface); + if (FAILED(hr)) + goto done; + IMFPresentationClock_AddRef(clock); + media_sink->clock = clock; + } + +done: + LeaveCriticalSection(&media_sink->cs); + return hr; } static HRESULT WINAPI media_sink_GetPresentationClock(IMFFinalizableMediaSink *iface, IMFPresentationClock **clock) @@ -913,6 +945,11 @@ static HRESULT WINAPI media_sink_Shutdown(IMFFinalizableMediaSink *iface) IMFMediaEventQueue_Shutdown(media_sink->event_queue); IMFByteStream_Close(media_sink->bytestream); + if (media_sink->clock) + { + IMFPresentationClock_RemoveClockStateSink(media_sink->clock, &media_sink->IMFClockStateSink_iface); + IMFPresentationClock_Release(media_sink->clock); + } media_sink->state = STATE_SHUTDOWN; -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/5159
From: Torge Matthies <tmatthies(a)codeweavers.com> --- dlls/winegstreamer/media_sink.c | 21 +++++++++++++++++++-- 1 file changed, 19 insertions(+), 2 deletions(-) diff --git a/dlls/winegstreamer/media_sink.c b/dlls/winegstreamer/media_sink.c index 0d76c6f30b9..db8fa2fe0c4 100644 --- a/dlls/winegstreamer/media_sink.c +++ b/dlls/winegstreamer/media_sink.c @@ -916,9 +916,26 @@ done: static HRESULT WINAPI media_sink_GetPresentationClock(IMFFinalizableMediaSink *iface, IMFPresentationClock **clock) { - FIXME("iface %p, clock %p stub!\n", iface, clock); + struct media_sink *media_sink = impl_from_IMFFinalizableMediaSink(iface); + HRESULT hr = S_OK; - return E_NOTIMPL; + TRACE("iface %p, clock %p\n", iface, clock); + + if (!clock) + return E_POINTER; + + EnterCriticalSection(&media_sink->cs); + + if (media_sink->state == STATE_SHUTDOWN) + hr = MF_E_SHUTDOWN; + else if (media_sink->clock) + IMFPresentationClock_AddRef(*clock = media_sink->clock); + else + hr = MF_E_NO_CLOCK; + + LeaveCriticalSection(&media_sink->cs); + + return hr; } static HRESULT WINAPI media_sink_Shutdown(IMFFinalizableMediaSink *iface) -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/5159
From: Torge Matthies <tmatthies(a)codeweavers.com> --- dlls/winegstreamer/media_sink.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/dlls/winegstreamer/media_sink.c b/dlls/winegstreamer/media_sink.c index db8fa2fe0c4..039acf81232 100644 --- a/dlls/winegstreamer/media_sink.c +++ b/dlls/winegstreamer/media_sink.c @@ -446,6 +446,7 @@ static HRESULT WINAPI stream_sink_type_handler_SetCurrentMediaType(IMFMediaTypeH static HRESULT WINAPI stream_sink_type_handler_GetCurrentMediaType(IMFMediaTypeHandler *iface, IMFMediaType **type) { struct stream_sink *stream_sink = impl_from_IMFMediaTypeHandler(iface); + HRESULT hr; TRACE("iface %p, type %p.\n", iface, type); @@ -454,9 +455,13 @@ static HRESULT WINAPI stream_sink_type_handler_GetCurrentMediaType(IMFMediaTypeH if (!stream_sink->type) return MF_E_NOT_INITIALIZED; - IMFMediaType_AddRef((*type = stream_sink->type)); + if (FAILED(hr = MFCreateMediaType(type))) + return hr; - return S_OK; + if (FAILED(hr = IMFMediaType_CopyAllItems(stream_sink->type, (IMFAttributes *)*type))) + IMFMediaType_Release(*type); + + return hr; } static HRESULT WINAPI stream_sink_type_handler_GetMajorType(IMFMediaTypeHandler *iface, GUID *type) -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/5159
From: Torge Matthies <tmatthies(a)codeweavers.com> --- dlls/mf/tests/mf.c | 5 ----- dlls/winegstreamer/media_sink.c | 11 +++++++++-- 2 files changed, 9 insertions(+), 7 deletions(-) diff --git a/dlls/mf/tests/mf.c b/dlls/mf/tests/mf.c index 2f288e854d7..d21d6841060 100644 --- a/dlls/mf/tests/mf.c +++ b/dlls/mf/tests/mf.c @@ -7826,12 +7826,9 @@ static void test_mpeg4_media_sink(void) ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); hr = IMFMediaTypeHandler_GetMajorType(type_handler, NULL); - todo_wine ok(hr == E_POINTER, "Unexpected hr %#lx.\n", hr); hr = IMFMediaTypeHandler_GetMajorType(type_handler, &guid); - todo_wine ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); - todo_wine ok(IsEqualGUID(&guid, &MFMediaType_Audio), "Unexpected major type.\n"); hr = IMFMediaTypeHandler_GetMediaTypeCount(type_handler, &count); @@ -7869,10 +7866,8 @@ static void test_mpeg4_media_sink(void) ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); hr = IMFMediaTypeHandler_GetMajorType(type_handler, NULL); - todo_wine ok(hr == E_POINTER, "Unexpected hr %#lx.\n", hr); hr = IMFMediaTypeHandler_GetMajorType(type_handler, &guid); - todo_wine ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); IMFStreamSink_Release(stream_sink); diff --git a/dlls/winegstreamer/media_sink.c b/dlls/winegstreamer/media_sink.c index 039acf81232..eba0d527741 100644 --- a/dlls/winegstreamer/media_sink.c +++ b/dlls/winegstreamer/media_sink.c @@ -466,9 +466,16 @@ static HRESULT WINAPI stream_sink_type_handler_GetCurrentMediaType(IMFMediaTypeH static HRESULT WINAPI stream_sink_type_handler_GetMajorType(IMFMediaTypeHandler *iface, GUID *type) { - FIXME("iface %p, type %p.\n", iface, type); + struct stream_sink *stream_sink = impl_from_IMFMediaTypeHandler(iface); - return E_NOTIMPL; + TRACE("iface %p, type %p.\n", iface, type); + + if (!type) + return E_POINTER; + if (!stream_sink->type) + return MF_E_NOT_INITIALIZED; + + return IMFMediaType_GetGUID(stream_sink->type, &MF_MT_MAJOR_TYPE, type); } static const IMFMediaTypeHandlerVtbl stream_sink_type_handler_vtbl = -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/5159
From: Torge Matthies <tmatthies(a)codeweavers.com> --- dlls/winegstreamer/gst_private.h | 2 + dlls/winegstreamer/main.c | 25 +++ dlls/winegstreamer/mfplat.c | 60 ++++++- dlls/winegstreamer/unix_private.h | 2 + dlls/winegstreamer/unixlib.h | 12 ++ dlls/winegstreamer/wg_format.c | 262 ++++++++++++++++++++++++++++++ dlls/winegstreamer/wg_parser.c | 4 + 7 files changed, 363 insertions(+), 4 deletions(-) diff --git a/dlls/winegstreamer/gst_private.h b/dlls/winegstreamer/gst_private.h index 2374fb6fddd..59893ff1bad 100644 --- a/dlls/winegstreamer/gst_private.h +++ b/dlls/winegstreamer/gst_private.h @@ -149,6 +149,8 @@ unsigned int wg_format_get_stride(const struct wg_format *format); bool wg_video_format_is_rgb(enum wg_video_format format); +HRESULT wg_create_aac_codec_data(uint32_t rate, unsigned int channels, uint8_t *out, size_t *out_size); + HRESULT aac_decoder_create(REFIID riid, void **ret); HRESULT h264_decoder_create(REFIID riid, void **ret); HRESULT video_processor_create(REFIID riid, void **ret); diff --git a/dlls/winegstreamer/main.c b/dlls/winegstreamer/main.c index 74a6eee5b1d..7ebd44ff0e8 100644 --- a/dlls/winegstreamer/main.c +++ b/dlls/winegstreamer/main.c @@ -600,6 +600,31 @@ HRESULT wg_muxer_finalize(wg_muxer_t muxer) return S_OK; } +HRESULT wg_create_aac_codec_data(uint32_t rate, unsigned int channels, uint8_t *out, size_t *out_size) +{ + struct wg_create_aac_codec_data_params params = + { + .rate = rate, + .channels = channels, + .buffer = out, + .size = *out_size, + .err_on = ERR_ON(quartz), + .warn_on = WARN_ON(quartz), + }; + NTSTATUS status; + + TRACE("rate %u, channels %u, out %p, out_size %p.\n", rate, channels, out, out_size); + + if ((status = WINE_UNIX_CALL(unix_wg_create_aac_codec_data, ¶ms))) + { + WARN("Failed to create AAC codec_data, status %#lx.\n", status); + return HRESULT_FROM_NT(status); + } + + *out_size = params.size; + return S_OK; +} + #define ALIGN(n, alignment) (((n) + (alignment) - 1) & ~((alignment) - 1)) unsigned int wg_format_get_stride(const struct wg_format *format) diff --git a/dlls/winegstreamer/mfplat.c b/dlls/winegstreamer/mfplat.c index e6d9fb9fd2c..3664ad295b2 100644 --- a/dlls/winegstreamer/mfplat.c +++ b/dlls/winegstreamer/mfplat.c @@ -25,6 +25,7 @@ #include "initguid.h" #include "d3d9types.h" #include "mfapi.h" +#include "mferror.h" #include "mmreg.h" #include "wine/debug.h" @@ -632,22 +633,73 @@ static void mf_media_type_to_wg_format_audio(IMFMediaType *type, const GUID *sub FIXME("Unrecognized audio subtype %s, depth %u.\n", debugstr_guid(subtype), depth); } +static HRESULT make_dummy_aac_user_data(IMFMediaType *type, BOOL raw, BYTE *out, size_t *len) +{ + UINT32 payload_type, profile_level, rate, channels; + HEAACWAVEFORMAT *user_data; + size_t cdlen = *len, hdrlen; + NTSTATUS status; + + if (!raw) + { + hdrlen = FIELD_OFFSET(HEAACWAVEFORMAT, pbAudioSpecificConfig) - FIELD_OFFSET(HEAACWAVEFORMAT, wfInfo.wPayloadType); + if (*len < hdrlen) + return E_NOT_SUFFICIENT_BUFFER; + + if (FAILED(IMFMediaType_GetUINT32(type, &MF_MT_AAC_PAYLOAD_TYPE, &payload_type))) + payload_type = 0; + else if (payload_type > 3) + return MF_E_INVALIDMEDIATYPE; + if (FAILED(IMFMediaType_GetUINT32(type, &MF_MT_AAC_AUDIO_PROFILE_LEVEL_INDICATION, &profile_level))) + profile_level = 0x29; + else if (profile_level > 0xFFFF) + return MF_E_INVALIDMEDIATYPE; + + user_data = (HEAACWAVEFORMAT *)(out - FIELD_OFFSET(HEAACWAVEFORMAT, wfInfo.wPayloadType)); + user_data->wfInfo.wPayloadType = payload_type; + user_data->wfInfo.wAudioProfileLevelIndication = profile_level; + out = user_data->pbAudioSpecificConfig; + cdlen -= hdrlen; + } + + if (FAILED(IMFMediaType_GetUINT32(type, &MF_MT_AUDIO_SAMPLES_PER_SECOND, &rate)) || + rate > 0xFFFFFF) + return MF_E_INVALIDMEDIATYPE; + if (FAILED(IMFMediaType_GetUINT32(type, &MF_MT_AUDIO_NUM_CHANNELS, &channels)) || + channels < 1 || channels > 8 || channels == 7) + return MF_E_INVALIDMEDIATYPE; + + status = wg_create_aac_codec_data(rate, channels, out, &cdlen); + if (status) + return HRESULT_FROM_NT(status); + + if (raw) + *len = cdlen; + else + *len = FIELD_OFFSET(HEAACWAVEFORMAT, pbAudioSpecificConfig[cdlen]) - FIELD_OFFSET(HEAACWAVEFORMAT, wfInfo.wPayloadType); + return S_OK; +} + static void mf_media_type_to_wg_format_audio_mpeg4(IMFMediaType *type, const GUID *subtype, struct wg_format *format) { BYTE buffer[sizeof(HEAACWAVEFORMAT) + 64]; HEAACWAVEFORMAT *wfx = (HEAACWAVEFORMAT *)buffer; UINT32 codec_data_size; - BOOL raw_aac; + BOOL raw_aac = IsEqualGUID(subtype, &MFAudioFormat_RAW_AAC); + size_t len = sizeof(buffer) - sizeof(wfx->wfInfo.wfx); wfx->wfInfo.wfx.cbSize = sizeof(buffer) - sizeof(wfx->wfInfo.wfx); if (FAILED(IMFMediaType_GetBlob(type, &MF_MT_USER_DATA, (BYTE *)(&wfx->wfInfo.wfx + 1), wfx->wfInfo.wfx.cbSize, &codec_data_size))) { - FIXME("Codec data is not set.\n"); - return; + if (FAILED(make_dummy_aac_user_data(type, raw_aac, (BYTE *)(&wfx->wfInfo.wfx + 1), &len))) + { + FIXME("Codec data is not set.\n"); + return; + } + codec_data_size = len; } - raw_aac = IsEqualGUID(subtype, &MFAudioFormat_RAW_AAC); if (!raw_aac) codec_data_size -= min(codec_data_size, sizeof(HEAACWAVEINFO) - sizeof(WAVEFORMATEX)); if (codec_data_size > sizeof(format->u.audio_mpeg4.codec_data)) diff --git a/dlls/winegstreamer/unix_private.h b/dlls/winegstreamer/unix_private.h index bb2cb864735..1ef07168c61 100644 --- a/dlls/winegstreamer/unix_private.h +++ b/dlls/winegstreamer/unix_private.h @@ -48,6 +48,8 @@ extern bool push_event(GstPad *pad, GstEvent *event); /* wg_format.c */ +extern NTSTATUS wg_create_aac_codec_data(void *args); + extern void wg_format_from_caps(struct wg_format *format, const GstCaps *caps); extern bool wg_format_compare(const struct wg_format *a, const struct wg_format *b); extern GstCaps *wg_format_to_caps(const struct wg_format *format); diff --git a/dlls/winegstreamer/unixlib.h b/dlls/winegstreamer/unixlib.h index 4ec9fce515e..85f7f89938b 100644 --- a/dlls/winegstreamer/unixlib.h +++ b/dlls/winegstreamer/unixlib.h @@ -412,6 +412,16 @@ struct wg_muxer_read_data_params UINT64 offset; }; +struct wg_create_aac_codec_data_params +{ + UINT32 rate; + UINT32 channels; + void *buffer; + UINT64 size; + UINT8 err_on; + UINT8 warn_on; +}; + enum unix_funcs { unix_wg_init_gstreamer, @@ -461,6 +471,8 @@ enum unix_funcs unix_wg_muxer_read_data, unix_wg_muxer_finalize, + unix_wg_create_aac_codec_data, + unix_wg_funcs_count, }; diff --git a/dlls/winegstreamer/wg_format.c b/dlls/winegstreamer/wg_format.c index 6da97685736..d59b665c770 100644 --- a/dlls/winegstreamer/wg_format.c +++ b/dlls/winegstreamer/wg_format.c @@ -34,6 +34,8 @@ #include <gst/video/video.h> #include <gst/audio/audio.h> +#include "ntstatus.h" +#define WIN32_NO_STATUS #include "winternl.h" #include "codecapi.h" #include "dshow.h" @@ -390,6 +392,266 @@ static void wg_format_from_caps_video_mpeg1(struct wg_format *format, const GstC format->u.video_mpeg1.fps_d = fps_d; } +struct wg_create_aac_codec_data_state +{ + pthread_mutex_t mutex; + pthread_cond_t cond; + struct wg_create_aac_codec_data_params *params; + BOOL got_caps, error; +}; + +static GstBusSyncReply bus_handler_cb(GstBus *bus, GstMessage *msg, gpointer user) +{ + struct wg_create_aac_codec_data_state *state = user; + gchar *dbg_info = NULL; + GError *err = NULL; + + GST_LOG("message type %s.", GST_MESSAGE_TYPE_NAME(msg)); + + switch (msg->type) + { + case GST_MESSAGE_ERROR: + gst_message_parse_error(msg, &err, &dbg_info); + if (state->params->err_on) + { + fprintf(stderr, "winegstreamer error: %s: %s\n", GST_OBJECT_NAME(msg->src), err->message); + fprintf(stderr, "winegstreamer error: %s: %s\n", GST_OBJECT_NAME(msg->src), dbg_info); + } + g_error_free(err); + g_free(dbg_info); + pthread_mutex_lock(&state->mutex); + state->error = true; + pthread_mutex_unlock(&state->mutex); + pthread_cond_signal(&state->cond); + break; + + case GST_MESSAGE_WARNING: + gst_message_parse_warning(msg, &err, &dbg_info); + if (state->params->warn_on) + { + fprintf(stderr, "winegstreamer warning: %s: %s\n", GST_OBJECT_NAME(msg->src), err->message); + fprintf(stderr, "winegstreamer warning: %s: %s\n", GST_OBJECT_NAME(msg->src), dbg_info); + } + g_error_free(err); + g_free(dbg_info); + break; + + default: + break; + } + gst_message_unref(msg); + return GST_BUS_DROP; +} + +static GstFlowReturn sink_chain_cb(GstPad *pad, GstObject *parent, GstBuffer *buffer) +{ + gst_buffer_unref(buffer); + return GST_FLOW_OK; +} + +static gboolean sink_event_cb(GstPad *pad, GstObject *parent, GstEvent *event) +{ + struct wg_create_aac_codec_data_state *state = gst_pad_get_element_private(pad); + + GST_LOG("state %p, type \"%s\".", state, GST_EVENT_TYPE_NAME(event)); + + switch (event->type) + { + case GST_EVENT_CAPS: + { + const GstStructure *structure = NULL; + const GValue *codec_data_value; + GstBuffer *codec_data; + GstMapInfo map; + GstCaps *caps; + + gst_event_parse_caps(event, &caps); + structure = gst_caps_get_structure(caps, 0); + if (!(codec_data_value = gst_structure_get_value(structure, "codec_data")) || + !(codec_data = gst_value_get_buffer(codec_data_value))) + { + GST_WARNING("Missing \"codec_data\" value in %" GST_PTR_FORMAT ".", caps); + break; + } + + gst_buffer_map(codec_data, &map, GST_MAP_READ); + if (map.size <= state->params->size) + { + state->params->size = map.size; + memcpy(state->params->buffer, map.data, map.size); + } + else + GST_WARNING("Too big codec_data value (%u) in %" GST_PTR_FORMAT ".", (UINT)map.size, caps); + gst_buffer_unmap(codec_data, &map); + + pthread_mutex_lock(&state->mutex); + state->got_caps = TRUE; + pthread_mutex_unlock(&state->mutex); + pthread_cond_signal(&state->cond); + break; + } + + default: + break; + } + gst_event_unref(event); + return TRUE; +} + +extern NTSTATUS wg_create_aac_codec_data(void *args) +{ + struct wg_create_aac_codec_data_params *params = args; + GstPad *encoder_sink_pad = NULL, *encoder_src_pad = NULL, *parser_sink_pad = NULL, + *parser_src_pad = NULL, *sink_pad = NULL; + struct wg_create_aac_codec_data_state state; + GstCaps *raw_caps = NULL, *aac_caps = NULL; + GstElement *encoder = NULL, *parser = NULL; + NTSTATUS status = STATUS_NO_MEMORY; + GstPadTemplate *sink_template; + GstElement *pipeline = NULL; + GstBuffer *buffer = NULL; + GstFlowReturn flowret; + GstSegment segment; + GstBus *bus = NULL; + + memset(&state, 0, sizeof(state)); + if (pthread_mutex_init(&state.mutex, NULL) || pthread_cond_init(&state.cond, NULL)) + goto out; + state.params = params; + + if (!(raw_caps = gst_caps_new_simple("audio/x-raw", + "format", G_TYPE_STRING, "S16LE", + "layout", G_TYPE_STRING, "interleaved", + "rate", G_TYPE_INT, params->rate, + "channels", G_TYPE_INT, params->channels, + "channel-mask", GST_TYPE_BITMASK, gst_audio_channel_get_fallback_mask(params->channels), + NULL))) + goto out; + + if (!(aac_caps = gst_caps_new_simple("audio/mpeg", + "mpegversion", G_TYPE_INT, 4, + "rate", G_TYPE_INT, params->rate, + "channels", G_TYPE_INT, params->channels, + "stream-format", G_TYPE_STRING, "raw", + "profile", G_TYPE_STRING, "lc", + "framed", G_TYPE_BOOLEAN, TRUE, + NULL))) + goto out; + + if (!(pipeline = gst_pipeline_new(NULL))) + goto out; + + if (!(sink_template = gst_pad_template_new("aac_codec_data_sink", GST_PAD_SINK, GST_PAD_ALWAYS, aac_caps))) + goto out; + sink_pad = gst_pad_new_from_template(sink_template, "aac_codec_data_sink"); + gst_object_unref(sink_template); + if (!sink_pad) + goto out; + GST_PAD_SET_ACCEPT_TEMPLATE(sink_pad); + gst_pad_use_fixed_caps(sink_pad); + gst_pad_set_element_private(sink_pad, &state); + gst_pad_set_chain_function(sink_pad, sink_chain_cb); + gst_pad_set_event_function(sink_pad, sink_event_cb); + + if (!(buffer = gst_buffer_new_allocate(NULL, 0, NULL))) + goto out; + buffer->dts = GST_CLOCK_TIME_NONE; + buffer->pts = 0; + buffer->duration = 0; + buffer->offset = 0; + buffer->offset_end = 0; + + status = STATUS_UNSUCCESSFUL; + + if (!(bus = gst_pipeline_get_bus(GST_PIPELINE(pipeline)))) + goto out; + gst_bus_set_sync_handler(bus, bus_handler_cb, &state, NULL); + + if (!(encoder = find_element(GST_ELEMENT_FACTORY_TYPE_ENCODER, raw_caps, aac_caps))) + goto out; + if (!gst_bin_add(GST_BIN(pipeline), encoder)) + { + gst_object_unref(encoder); + goto out; + } + if (!gst_element_sync_state_with_parent(encoder) + || !(encoder_sink_pad = gst_element_get_static_pad(encoder, "sink")) + || !(encoder_src_pad = gst_element_get_static_pad(encoder, "src"))) + goto out; + + if (!(parser = find_element(GST_ELEMENT_FACTORY_TYPE_PARSER, aac_caps, NULL))) + goto out; + if (!gst_bin_add(GST_BIN(pipeline), parser)) + { + gst_object_unref(parser); + goto out; + } + if (!gst_element_sync_state_with_parent(parser) + || !(parser_sink_pad = gst_element_get_static_pad(parser, "sink")) + || !(parser_src_pad = gst_element_get_static_pad(parser, "src"))) + goto out; + + if (!gst_element_link(encoder, parser) + || !link_src_to_sink(parser_src_pad, sink_pad) + || !gst_pad_set_active(parser_src_pad, 1) + || !gst_pad_set_active(parser_sink_pad, 1) + || !gst_pad_set_active(encoder_src_pad, 1) + || !gst_pad_set_active(encoder_sink_pad, 1) + || !gst_pad_set_active(sink_pad, 1)) + goto out; + + gst_segment_init(&segment, GST_FORMAT_TIME); + if (gst_element_set_state(pipeline, GST_STATE_READY) == GST_STATE_CHANGE_FAILURE + || gst_element_get_state(pipeline, NULL, NULL, -1) == GST_STATE_CHANGE_FAILURE + || !gst_pad_send_event(encoder_sink_pad, gst_event_new_stream_start("aac_codec_data")) + || !gst_pad_send_event(encoder_sink_pad, gst_event_new_caps(raw_caps)) + || !gst_pad_send_event(encoder_sink_pad, gst_event_new_segment(&segment)) + || gst_element_set_state(pipeline, GST_STATE_PLAYING) == GST_STATE_CHANGE_FAILURE + || gst_element_get_state(pipeline, NULL, NULL, -1) == GST_STATE_CHANGE_FAILURE) + goto out; + + flowret = gst_pad_chain(encoder_sink_pad, buffer); + buffer = NULL; + if (flowret == GST_FLOW_ERROR || !gst_pad_send_event(encoder_sink_pad, gst_event_new_eos())) + goto out; + + pthread_mutex_lock(&state.mutex); + while (!state.got_caps && !state.error) + pthread_cond_wait(&state.cond, &state.mutex); + pthread_mutex_unlock(&state.mutex); + +out: + if (pipeline && gst_element_set_state(pipeline, GST_STATE_NULL) != GST_STATE_CHANGE_FAILURE) + gst_element_get_state(pipeline, NULL, NULL, -1); + + if (parser_src_pad) gst_object_unref(parser_src_pad); + if (parser_sink_pad) gst_object_unref(parser_sink_pad); + if (encoder_src_pad) gst_object_unref(encoder_src_pad); + if (encoder_sink_pad) gst_object_unref(encoder_sink_pad); + if (buffer) gst_buffer_unref(buffer); + if (pipeline) gst_object_unref(pipeline); + if (aac_caps) gst_caps_unref(aac_caps); + if (raw_caps) gst_caps_unref(raw_caps); + if (bus) + { + gst_bus_set_sync_handler(bus, NULL, NULL, NULL); + gst_object_unref(bus); + } + if (sink_pad) + { + gst_pad_set_event_function(sink_pad, NULL); + gst_pad_set_chain_function(sink_pad, NULL); + gst_pad_set_element_private(sink_pad, NULL); + gst_object_unref(sink_pad); + } + pthread_mutex_destroy(&state.mutex); + pthread_cond_destroy(&state.cond); + if (state.got_caps) + return STATUS_SUCCESS; + GST_ERROR("Failed creating AAC codec data, status %#x.", (unsigned int)status); + return status; +} + void wg_format_from_caps(struct wg_format *format, const GstCaps *caps) { const GstStructure *structure = gst_caps_get_structure(caps, 0); diff --git a/dlls/winegstreamer/wg_parser.c b/dlls/winegstreamer/wg_parser.c index b0a344d5186..ea7bfed41c3 100644 --- a/dlls/winegstreamer/wg_parser.c +++ b/dlls/winegstreamer/wg_parser.c @@ -1928,6 +1928,8 @@ const unixlib_entry_t __wine_unix_call_funcs[] = X(wg_muxer_push_sample), X(wg_muxer_read_data), X(wg_muxer_finalize), + + X(wg_create_aac_codec_data), }; C_ASSERT(ARRAYSIZE(__wine_unix_call_funcs) == unix_wg_funcs_count); @@ -2261,6 +2263,8 @@ const unixlib_entry_t __wine_unix_call_wow64_funcs[] = X64(wg_muxer_push_sample), X64(wg_muxer_read_data), X(wg_muxer_finalize), + + X(wg_create_aac_codec_data), }; C_ASSERT(ARRAYSIZE(__wine_unix_call_wow64_funcs) == unix_wg_funcs_count); -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/5159
From: Torge Matthies <tmatthies(a)codeweavers.com> --- dlls/mf/tests/mf.c | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/dlls/mf/tests/mf.c b/dlls/mf/tests/mf.c index d21d6841060..3ffbccf6a0c 100644 --- a/dlls/mf/tests/mf.c +++ b/dlls/mf/tests/mf.c @@ -7602,8 +7602,9 @@ static void test_MFRequireProtectedEnvironment(void) static void test_mpeg4_media_sink(void) { - IMFMediaSink *sink = NULL, *sink2 = NULL, *sink_audio = NULL, *sink_video = NULL, *sink_empty = NULL; - IMFByteStream *bytestream, *bytestream_audio, *bytestream_video, *bytestream_empty; + IMFMediaSink *sink = NULL, *sink2 = NULL, *sink_audio = NULL, *sink_video = NULL, *sink_empty = NULL, + *sink_no_user_data = NULL; + IMFByteStream *bytestream, *bytestream_audio, *bytestream_video, *bytestream_empty, *bytestream_no_user_data; DWORD id, count, flags, width = 96, height = 96; IMFMediaType *audio_type, *video_type, *media_type; IMFMediaTypeHandler *type_handler = NULL; @@ -7634,8 +7635,6 @@ static void test_mpeg4_media_sink(void) ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); hr = IMFMediaType_SetUINT32(audio_type, &MF_MT_AAC_PAYLOAD_TYPE, 0); ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); - hr = IMFMediaType_SetBlob(audio_type, &MF_MT_USER_DATA, test_aac_codec_data, sizeof(test_aac_codec_data)); - ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); hr = IMFMediaType_SetGUID(video_type, &MF_MT_MAJOR_TYPE, &MFMediaType_Video); ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); @@ -7649,6 +7648,8 @@ static void test_mpeg4_media_sink(void) test_h264_sequence_header, sizeof(test_h264_sequence_header)); ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + hr = MFCreateTempFile(MF_ACCESSMODE_WRITE, MF_OPENMODE_DELETE_IF_EXIST, 0, &bytestream_no_user_data); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); hr = MFCreateTempFile(MF_ACCESSMODE_WRITE, MF_OPENMODE_DELETE_IF_EXIST, 0, &bytestream_audio); ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); hr = MFCreateTempFile(MF_ACCESSMODE_WRITE, MF_OPENMODE_DELETE_IF_EXIST, 0, &bytestream_video); @@ -7670,6 +7671,15 @@ static void test_mpeg4_media_sink(void) hr = MFCreateMPEG4MediaSink(bytestream_empty, NULL, NULL, &sink_empty); ok(hr == S_OK || broken(hr == E_INVALIDARG), "Unexpected hr %#lx.\n", hr); + hr = MFCreateMPEG4MediaSink(bytestream_no_user_data, NULL, audio_type, &sink_no_user_data); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + + IMFMediaSink_Release(sink_no_user_data); + IMFByteStream_Release(bytestream_no_user_data); + + hr = IMFMediaType_SetBlob(audio_type, &MF_MT_USER_DATA, test_aac_codec_data, sizeof(test_aac_codec_data)); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + hr = MFCreateMPEG4MediaSink(bytestream_audio, NULL, audio_type, &sink_audio); ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/5159
From: Torge Matthies <tmatthies(a)codeweavers.com> --- dlls/winegstreamer/media_sink.c | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/dlls/winegstreamer/media_sink.c b/dlls/winegstreamer/media_sink.c index eba0d527741..6e9727aa5a9 100644 --- a/dlls/winegstreamer/media_sink.c +++ b/dlls/winegstreamer/media_sink.c @@ -599,7 +599,12 @@ static HRESULT media_sink_start(struct media_sink *media_sink) media_sink->state = STATE_STARTED; - return media_sink_queue_stream_event(media_sink, MEStreamSinkStarted); + if (FAILED(hr = media_sink_queue_stream_event(media_sink, MEStreamSinkStarted))) + TRACE("Queueing MEStreamSinkStarted failed\n"); + else if (FAILED(hr = media_sink_queue_stream_event(media_sink, MEStreamSinkRequestSample))) + TRACE("Queueing MEStreamSinkRequestSample failed\n"); + + return hr; } static HRESULT media_sink_stop(struct media_sink *media_sink) @@ -617,6 +622,7 @@ static HRESULT media_sink_pause(struct media_sink *media_sink) static HRESULT media_sink_process(struct media_sink *media_sink, IMFSample *sample, UINT32 stream_id) { + struct stream_sink *stream_sink = NULL; wg_muxer_t muxer = media_sink->muxer; struct wg_sample *wg_sample; LONGLONG time, duration; @@ -649,6 +655,18 @@ static HRESULT media_sink_process(struct media_sink *media_sink, IMFSample *samp hr = wg_muxer_push_sample(muxer, wg_sample, stream_id); wg_sample_release(wg_sample); + if (SUCCEEDED(hr)) + { + LIST_FOR_EACH_ENTRY(stream_sink, &media_sink->stream_sinks, struct stream_sink, entry) + { + if (stream_sink->id == stream_id) + { + hr = IMFMediaEventQueue_QueueEventParamVar(stream_sink->event_queue, MEStreamSinkRequestSample, &GUID_NULL, S_OK, NULL); + break; + } + } + } + return hr; } -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/5159
From: Torge Matthies <tmatthies(a)codeweavers.com> --- dlls/mfreadwrite/writer.c | 2 ++ dlls/winegstreamer/gst_private.h | 1 + dlls/winegstreamer/main.c | 3 +++ dlls/winegstreamer/media_sink.c | 23 ++++++++++++++++++++ dlls/winegstreamer/winegstreamer_classes.idl | 7 ++++++ 5 files changed, 36 insertions(+) diff --git a/dlls/mfreadwrite/writer.c b/dlls/mfreadwrite/writer.c index c708664d2fc..f4ef3bc12d4 100644 --- a/dlls/mfreadwrite/writer.c +++ b/dlls/mfreadwrite/writer.c @@ -1018,6 +1018,7 @@ static HRESULT sink_writer_get_sink_factory_class(const WCHAR *url, IMFAttribute { L".mp3", &MFTranscodeContainerType_MP3 }, { L".wav", &MFTranscodeContainerType_WAVE }, { L".avi", &MFTranscodeContainerType_AVI }, + { L".aac", &MFTranscodeContainerType_ADTS }, }; static const struct { @@ -1029,6 +1030,7 @@ static HRESULT sink_writer_get_sink_factory_class(const WCHAR *url, IMFAttribute { &MFTranscodeContainerType_MP3, &CLSID_MFMP3SinkClassFactory }, { &MFTranscodeContainerType_WAVE, &CLSID_MFWAVESinkClassFactory }, { &MFTranscodeContainerType_AVI, &CLSID_MFAVISinkClassFactory }, + { &MFTranscodeContainerType_ADTS, &CLSID_MFADTSSinkClassFactory }, }; const WCHAR *extension; GUID container; diff --git a/dlls/winegstreamer/gst_private.h b/dlls/winegstreamer/gst_private.h index 59893ff1bad..d317b2eefc9 100644 --- a/dlls/winegstreamer/gst_private.h +++ b/dlls/winegstreamer/gst_private.h @@ -115,6 +115,7 @@ HRESULT resampler_create(IUnknown *outer, IUnknown **out); HRESULT color_convert_create(IUnknown *outer, IUnknown **out); HRESULT mp3_sink_class_factory_create(IUnknown *outer, IUnknown **out); HRESULT mpeg4_sink_class_factory_create(IUnknown *outer, IUnknown **out); +HRESULT adts_sink_class_factory_create(IUnknown *outer, IUnknown **out); bool amt_from_wg_format(AM_MEDIA_TYPE *mt, const struct wg_format *format, bool wm); bool amt_to_wg_format(const AM_MEDIA_TYPE *mt, struct wg_format *format); diff --git a/dlls/winegstreamer/main.c b/dlls/winegstreamer/main.c index 7ebd44ff0e8..612632108ec 100644 --- a/dlls/winegstreamer/main.c +++ b/dlls/winegstreamer/main.c @@ -784,6 +784,7 @@ static struct class_factory resampler_cf = {{&class_factory_vtbl}, resampler_cre static struct class_factory color_convert_cf = {{&class_factory_vtbl}, color_convert_create}; static struct class_factory mp3_sink_class_factory_cf = {{&class_factory_vtbl}, mp3_sink_class_factory_create}; static struct class_factory mpeg4_sink_class_factory_cf = {{&class_factory_vtbl}, mpeg4_sink_class_factory_create}; +static struct class_factory adts_sink_class_factory_cf = {{&class_factory_vtbl}, adts_sink_class_factory_create}; HRESULT WINAPI DllGetClassObject(REFCLSID clsid, REFIID iid, void **out) { @@ -824,6 +825,8 @@ HRESULT WINAPI DllGetClassObject(REFCLSID clsid, REFIID iid, void **out) factory = &mp3_sink_class_factory_cf; else if (IsEqualGUID(clsid, &CLSID_MFMPEG4SinkClassFactory)) factory = &mpeg4_sink_class_factory_cf; + else if (IsEqualGUID(clsid, &CLSID_MFADTSSinkClassFactory)) + factory = &adts_sink_class_factory_cf; else { FIXME("%s not implemented, returning CLASS_E_CLASSNOTAVAILABLE.\n", debugstr_guid(clsid)); diff --git a/dlls/winegstreamer/media_sink.c b/dlls/winegstreamer/media_sink.c index 6e9727aa5a9..78f8249692d 100644 --- a/dlls/winegstreamer/media_sink.c +++ b/dlls/winegstreamer/media_sink.c @@ -1421,6 +1421,14 @@ static HRESULT WINAPI mpeg4_sink_class_factory_CreateMediaSink(IMFSinkClassFacto return sink_class_factory_create_media_sink(iface, bytestream, format, video_type, audio_type, out); } +static HRESULT WINAPI adts_sink_class_factory_CreateMediaSink(IMFSinkClassFactory *iface, IMFByteStream *bytestream, + IMFMediaType *video_type, IMFMediaType *audio_type, IMFMediaSink **out) +{ + const char *format = "application/x-gst-av-adts"; + + return sink_class_factory_create_media_sink(iface, bytestream, format, video_type, audio_type, out); +} + static const IMFSinkClassFactoryVtbl mp3_sink_class_factory_vtbl = { sink_class_factory_QueryInterface, @@ -1437,8 +1445,17 @@ static const IMFSinkClassFactoryVtbl mpeg4_sink_class_factory_vtbl = mpeg4_sink_class_factory_CreateMediaSink, }; +static const IMFSinkClassFactoryVtbl aac_sink_class_factory_vtbl = +{ + sink_class_factory_QueryInterface, + sink_class_factory_AddRef, + sink_class_factory_Release, + adts_sink_class_factory_CreateMediaSink, +}; + static IMFSinkClassFactory mp3_sink_class_factory = { &mp3_sink_class_factory_vtbl }; static IMFSinkClassFactory mpeg4_sink_class_factory = { &mpeg4_sink_class_factory_vtbl }; +static IMFSinkClassFactory aac_sink_class_factory = { &aac_sink_class_factory_vtbl }; HRESULT mp3_sink_class_factory_create(IUnknown *outer, IUnknown **out) { @@ -1451,3 +1468,9 @@ HRESULT mpeg4_sink_class_factory_create(IUnknown *outer, IUnknown **out) *out = (IUnknown *)&mpeg4_sink_class_factory; return S_OK; } + +HRESULT adts_sink_class_factory_create(IUnknown *outer, IUnknown **out) +{ + *out = (IUnknown *)&aac_sink_class_factory; + return S_OK; +} diff --git a/dlls/winegstreamer/winegstreamer_classes.idl b/dlls/winegstreamer/winegstreamer_classes.idl index bb727ca8645..ec2103c5f26 100644 --- a/dlls/winegstreamer/winegstreamer_classes.idl +++ b/dlls/winegstreamer/winegstreamer_classes.idl @@ -131,3 +131,10 @@ coclass MFMP3SinkClassFactory {} uuid(a22c4fc7-6e91-4e1d-89e9-53b2667b72ba) ] coclass MFMPEG4SinkClassFactory {} + +[ + helpstring("MF ADTS Sink Class Factory"), + threading(both), + uuid(d7ca55ab-5022-4db3-a599-abafa358e6f3) +] +coclass MFADTSSinkClassFactory {} -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/5159
From: Torge Matthies <tmatthies(a)codeweavers.com> --- dlls/mf/tests/mf.c | 195 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 195 insertions(+) diff --git a/dlls/mf/tests/mf.c b/dlls/mf/tests/mf.c index 3ffbccf6a0c..b4380d440fa 100644 --- a/dlls/mf/tests/mf.c +++ b/dlls/mf/tests/mf.c @@ -5786,6 +5786,200 @@ done: ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); } +#define wait_media_event_generator(a, b, c, d, e) wait_media_event_generator_(__LINE__, a, b, c, d, e) +static HRESULT wait_media_event_generator_(int line, IMFMediaEventGenerator *generator, IMFAsyncCallback *callback, + MediaEventType expect_type, DWORD timeout, PROPVARIANT *value) +{ + struct test_callback *impl = impl_from_IMFAsyncCallback(callback); + MediaEventType type; + HRESULT hr, status; + DWORD ret; + GUID guid; + + do + { + hr = IMFMediaEventGenerator_BeginGetEvent(generator, &impl->IMFAsyncCallback_iface, (IUnknown *)generator); + ok_(__FILE__, line)(hr == S_OK, "Unexpected hr %#lx.\n", hr); + ret = WaitForSingleObject(impl->event, timeout); + ok_(__FILE__, line)(ret == WAIT_OBJECT_0, "WaitForSingleObject returned %lu\n", ret); + hr = IMFMediaEvent_GetType(impl->media_event, &type); + ok_(__FILE__, line)(hr == S_OK, "Unexpected hr %#lx.\n", hr); + } while (type != expect_type); + + ok_(__FILE__, line)(type == expect_type, "got type %lu\n", type); + + hr = IMFMediaEvent_GetExtendedType(impl->media_event, &guid); + ok_(__FILE__, line)(hr == S_OK, "Unexpected hr %#lx.\n", hr); + ok_(__FILE__, line)(IsEqualGUID(&guid, &GUID_NULL), "got extended type %s\n", debugstr_guid(&guid)); + + hr = IMFMediaEvent_GetValue(impl->media_event, value); + ok_(__FILE__, line)(hr == S_OK, "Unexpected hr %#lx.\n", hr); + + hr = IMFMediaEvent_GetStatus(impl->media_event, &status); + ok_(__FILE__, line)(hr == S_OK, "Unexpected hr %#lx.\n", hr); + + return status; +} + +static void load_resource(const WCHAR *filename, const BYTE **data, DWORD *length) +{ + HRSRC resource = FindResourceW(NULL, filename, (const WCHAR *)RT_RCDATA); + ok(resource != 0, "FindResourceW failed, error %lu\n", GetLastError()); + *data = LockResource(LoadResource(GetModuleHandleW(NULL), resource)); + *length = SizeofResource(GetModuleHandleW(NULL), resource); +} + +static void test_media_sink_events(void) +{ + IMFPresentationTimeSource *time_source; + IMFPresentationClock *clock, *clock2; + IMFMediaEventGenerator *eg; + IMFAsyncCallback *callback; + IMFByteStream *bytestream; + IMFMediaType *audio_type; + IMFMediaTypeHandler *mth; + const BYTE *aacenc_data; + IMFSample *input_sample; + MF_ATTRIBUTE_TYPE type; + IMFStreamSink *stream; + ULONG aacenc_data_len; + IMFMediaEvent *event; + PROPVARIANT propvar; + IMFMediaSink *sink; + HRESULT hr; + + hr = MFStartup(MF_VERSION, MFSTARTUP_FULL); + ok(hr == S_OK, "Failed to start up, hr %#lx.\n", hr); + + callback = create_test_callback(TRUE); + + hr = MFCreateTempFile(MF_ACCESSMODE_WRITE, MF_OPENMODE_DELETE_IF_EXIST, 0, &bytestream); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + + hr = MFCreateMediaType(&audio_type); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + + hr = IMFMediaType_SetGUID(audio_type, &MF_MT_MAJOR_TYPE, &MFMediaType_Audio); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + hr = IMFMediaType_SetGUID(audio_type, &MF_MT_SUBTYPE, &MFAudioFormat_AAC); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + hr = IMFMediaType_SetUINT32(audio_type, &MF_MT_AUDIO_NUM_CHANNELS, 1); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + hr = IMFMediaType_SetUINT32(audio_type, &MF_MT_AUDIO_BITS_PER_SAMPLE, 16); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + hr = IMFMediaType_SetUINT32(audio_type, &MF_MT_AUDIO_SAMPLES_PER_SECOND, 44100); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + hr = IMFMediaType_SetUINT32(audio_type, &MF_MT_AUDIO_AVG_BYTES_PER_SECOND, 12000); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + hr = IMFMediaType_SetUINT32(audio_type, &MF_MT_AAC_AUDIO_PROFILE_LEVEL_INDICATION, 41); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + hr = IMFMediaType_SetUINT32(audio_type, &MF_MT_AAC_PAYLOAD_TYPE, 0); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + + hr = MFCreateADTSMediaSink(bytestream, audio_type, &sink); + IMFMediaType_Release(audio_type); + if (hr == REGDB_E_CLASSNOTREG) + { + skip("ADTS media sink is not supported, skipping tests.\n"); + goto done; + } + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + + hr = MFCreatePresentationClock(&clock); + ok(hr == S_OK, "Failed to create clock object, hr %#lx.\n", hr); + + hr = IMFMediaSink_SetPresentationClock(sink, clock); + ok(hr == S_OK, "Failed to set presentation clock, hr %#lx.\n", hr); + + hr = MFCreateSystemTimeSource(&time_source); + ok(hr == S_OK, "Failed to create time source, hr %#lx.\n", hr); + + hr = IMFPresentationClock_SetTimeSource(clock, time_source); + ok(hr == S_OK, "Failed to set time source, hr %#lx.\n", hr); + IMFPresentationTimeSource_Release(time_source); + + hr = IMFMediaSink_GetStreamSinkByIndex(sink, 0, &stream); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + + hr = IMFStreamSink_GetMediaTypeHandler(stream, &mth); + ok(hr == S_OK, "Failed to get media type handler, hr %#lx.\n", hr); + + hr = IMFMediaTypeHandler_GetCurrentMediaType(mth, &audio_type); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + + hr = IMFMediaType_GetItemType(audio_type, &MF_MT_USER_DATA, &type); + ok(hr == MF_E_ATTRIBUTENOTFOUND, "Unexpected hr %#lx.\n", hr); + + IMFMediaType_Release(audio_type); + IMFMediaTypeHandler_Release(mth); + + hr = IMFStreamSink_QueryInterface(stream, &IID_IMFMediaEventGenerator, (void **)&eg); + ok(hr == S_OK, "Failed to get interface, hr %#lx.\n", hr); + + hr = IMFMediaEventGenerator_GetEvent(eg, MF_EVENT_FLAG_NO_WAIT, &event); + ok(hr == MF_E_NO_EVENTS_AVAILABLE, "Unexpected hr %#lx.\n", hr); + + hr = IMFPresentationClock_Start(clock, 0); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + + hr = wait_media_event_generator(eg, callback, MEStreamSinkStarted, 1000, &propvar); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + ok(propvar.vt == VT_EMPTY, "got vt %u\n", propvar.vt); + PropVariantClear(&propvar); + + hr = wait_media_event_generator(eg, callback, MEStreamSinkRequestSample, 1000, &propvar); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + ok(propvar.vt == VT_EMPTY, "got vt %u\n", propvar.vt); + PropVariantClear(&propvar); + + hr = IMFMediaEventGenerator_GetEvent(eg, MF_EVENT_FLAG_NO_WAIT, &event); + ok(hr == MF_E_NO_EVENTS_AVAILABLE, "Unexpected hr %#lx.\n", hr); + + load_resource(L"aacencdata.bin", &aacenc_data, &aacenc_data_len); + ok(aacenc_data_len == 24861, "got length %lu\n", aacenc_data_len); + + input_sample = create_sample(aacenc_data + sizeof(DWORD), *(DWORD *)aacenc_data); + + hr = IMFStreamSink_ProcessSample(stream, input_sample); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + + hr = wait_media_event_generator(eg, callback, MEStreamSinkRequestSample, 1000, &propvar); + ok(hr == S_OK, "Unexpected hr %#lx.\n", hr); + ok(propvar.vt == VT_EMPTY, "got vt %u\n", propvar.vt); + PropVariantClear(&propvar); + + hr = IMFMediaEventGenerator_GetEvent(eg, MF_EVENT_FLAG_NO_WAIT, &event); + ok(hr == MF_E_NO_EVENTS_AVAILABLE, "Unexpected hr %#lx.\n", hr); + + /* Shutdown */ + EXPECT_REF(clock, 2); + hr = IMFMediaSink_Shutdown(sink); + ok(hr == S_OK, "Failed to shut down, hr %#lx.\n", hr); + EXPECT_REF(clock, 1); + + hr = IMFMediaSink_Shutdown(sink); + ok(hr == MF_E_SHUTDOWN, "Unexpected hr %#lx.\n", hr); + + clock2 = NULL; + hr = IMFMediaSink_GetPresentationClock(sink, &clock2); + ok(hr == MF_E_SHUTDOWN, "Unexpected hr %#lx.\n", hr); + ok(!clock2, "Unexpected clock %p.\n", clock2); + + hr = IMFMediaSink_SetPresentationClock(sink, clock); + ok(hr == MF_E_SHUTDOWN, "Unexpected hr %#lx.\n", hr); + + IMFMediaEventGenerator_Release(eg); + IMFStreamSink_Release(stream); + IMFPresentationClock_Release(clock); + IMFMediaSink_Release(sink); +done: + IMFAsyncCallback_Release(callback); + IMFByteStream_Release(bytestream); + + hr = MFShutdown(); + ok(hr == S_OK, "Failed to shut down, hr %#lx.\n", hr); +} + static void test_quality_manager(void) { IMFPresentationClock *clock; @@ -8267,6 +8461,7 @@ START_TEST(mf) test_sample_grabber_is_mediatype_supported(); test_sample_grabber_orientation(MFVideoFormat_RGB32); test_sample_grabber_orientation(MFVideoFormat_NV12); + test_media_sink_events(); test_quality_manager(); test_sar(); test_evr(); -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/5159
From: Torge Matthies <tmatthies(a)codeweavers.com> --- dlls/mfplat/mediatype.c | 4 ---- dlls/mfplat/tests/mfplat.c | 10 ++++++++-- 2 files changed, 8 insertions(+), 6 deletions(-) diff --git a/dlls/mfplat/mediatype.c b/dlls/mfplat/mediatype.c index e07cc2339c1..08fd863c2a0 100644 --- a/dlls/mfplat/mediatype.c +++ b/dlls/mfplat/mediatype.c @@ -2996,11 +2996,7 @@ HRESULT WINAPI MFCreateWaveFormatExFromMFMediaType(IMFMediaType *mediatype, WAVE return E_INVALIDARG; if (FAILED(hr = IMFMediaType_GetBlobSize(mediatype, &MF_MT_USER_DATA, &user_size))) - { - if (!IsEqualGUID(&subtype, &MFAudioFormat_PCM) && !IsEqualGUID(&subtype, &MFAudioFormat_Float)) - return hr; user_size = 0; - } if (flags == MFWaveFormatExConvertFlag_ForceExtensible) extra_size = sizeof(WAVEFORMATEXTENSIBLE) - sizeof(*format); diff --git a/dlls/mfplat/tests/mfplat.c b/dlls/mfplat/tests/mfplat.c index 97f5dafef86..bb5573f2ab5 100644 --- a/dlls/mfplat/tests/mfplat.c +++ b/dlls/mfplat/tests/mfplat.c @@ -83,6 +83,8 @@ DEFINE_MEDIATYPE_GUID(MFVideoFormat_IMC2, MAKEFOURCC('I','M','C','2')); DEFINE_MEDIATYPE_GUID(MFVideoFormat_IMC3, MAKEFOURCC('I','M','C','3')); DEFINE_MEDIATYPE_GUID(MFVideoFormat_IMC4, MAKEFOURCC('I','M','C','4')); +DEFINE_MEDIATYPE_GUID(MFAudioFormat_RAW_AAC,WAVE_FORMAT_RAW_AAC1); + DEFINE_MEDIATYPE_GUID(MEDIASUBTYPE_AVC1,MAKEFOURCC('A','V','C','1')); DEFINE_MEDIATYPE_GUID(MEDIASUBTYPE_MP42,MAKEFOURCC('M','P','4','2')); DEFINE_MEDIATYPE_GUID(MEDIASUBTYPE_mp42,MAKEFOURCC('m','p','4','2')); @@ -5132,8 +5134,12 @@ static void test_MFCreateWaveFormatExFromMFMediaType(void) } wave_fmt_tests[] = { - { &MFAudioFormat_PCM, WAVE_FORMAT_PCM, }, - { &MFAudioFormat_Float, WAVE_FORMAT_IEEE_FLOAT, }, + { &MFAudioFormat_PCM, WAVE_FORMAT_PCM, }, + { &MFAudioFormat_Float, WAVE_FORMAT_IEEE_FLOAT, }, + { &MFAudioFormat_AAC, WAVE_FORMAT_MPEG_HEAAC, }, + { &MFAudioFormat_RAW_AAC, WAVE_FORMAT_RAW_AAC1, }, + { &MFAudioFormat_MP3, WAVE_FORMAT_MPEGLAYER3, }, + { &MFAudioFormat_Opus, WAVE_FORMAT_OPUS, }, }; WAVEFORMATEXTENSIBLE *format_ext; IMFMediaType *mediatype; -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/5159
From: Torge Matthies <tmatthies(a)codeweavers.com> There are tests for this in dlls/mf, however they are gated behind IMFMediaSink::GetCharacteristics succeeding and the characteristics not containing MEDIASINK_FIXED_STREAMS. --- dlls/winegstreamer/media_sink.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/dlls/winegstreamer/media_sink.c b/dlls/winegstreamer/media_sink.c index 78f8249692d..2300046aaeb 100644 --- a/dlls/winegstreamer/media_sink.c +++ b/dlls/winegstreamer/media_sink.c @@ -794,6 +794,9 @@ static HRESULT WINAPI media_sink_AddStreamSink(IMFFinalizableMediaSink *iface, D TRACE("iface %p, stream_sink_id %#lx, media_type %p, stream_sink %p.\n", iface, stream_sink_id, media_type, stream_sink); + if (stream_sink) + *stream_sink = NULL; + EnterCriticalSection(&media_sink->cs); if (media_sink_get_stream_sink_by_id(media_sink, stream_sink_id)) -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/5159
From: Torge Matthies <tmatthies(a)codeweavers.com> I have seen pipelines with a vaapipostproc element, which makes the caps non-fixed. wg_format_from_caps calls gst_audio_info_from_caps, which expects fixed caps and asserts if they are not fixed. This does not result in an error in release builds, but we should still avoid it. --- dlls/winegstreamer/wg_parser.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dlls/winegstreamer/wg_parser.c b/dlls/winegstreamer/wg_parser.c index ea7bfed41c3..702538ac9e1 100644 --- a/dlls/winegstreamer/wg_parser.c +++ b/dlls/winegstreamer/wg_parser.c @@ -976,7 +976,7 @@ static void pad_added_cb(GstElement *element, GstPad *pad, gpointer user) if (!(stream = create_stream(parser))) return; - caps = gst_pad_query_caps(pad, NULL); + caps = gst_caps_fixate(gst_pad_query_caps(pad, NULL)); wg_format_from_caps(&stream->codec_format, caps); gst_caps_unref(caps); -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/5159
Hi, It looks like your patch introduced the new failures shown below. Please investigate and fix them before resubmitting your patch. If they are not new, fixing them anyway would help a lot. Otherwise please ask for the known failures list to be updated. The tests also ran into some preexisting test failures. If you know how to fix them that would be helpful. See the TestBot job for the details: The full results can be found at: https://testbot.winehq.org/JobDetails.pl?Key=144234 Your paranoid android. === debian11 (32 bit report) === mf: mf: Timeout === debian11 (32 bit ar:MA report) === mf: mf: Timeout === debian11 (32 bit de report) === mf: mf: Timeout === debian11 (32 bit fr report) === mf: mf: Timeout === debian11 (32 bit he:IL report) === mf: mf: Timeout === debian11 (32 bit hi:IN report) === mf: mf: Timeout === debian11 (32 bit zh:CN report) === mf: mf: Timeout === debian11b (32 bit WoW report) === mf: mf: Timeout === debian11b (64 bit WoW report) === mf: mf: Timeout transform.c:2385: Test failed: aacdec: "MF_MT_USER_DATA": SetInputType returned 0. transform.c:2385: Test failed: aacdec: "MF_MT_USER_DATA": SetInputType returned 0.
On Wed Mar 20 13:55:43 2024 +0000, Torge Matthies wrote:
Correction to this: Seems like just setting the sink pad caps and reading the src pad caps doesn't work, I need to at least encode a 0-sized buffer and have an aacparse element parse the output. It's using Gstreamer for the AAC codec data now.
-- https://gitlab.winehq.org/wine/wine/-/merge_requests/5159#note_65500
On Wed Mar 20 13:55:53 2024 +0000, Torge Matthies wrote:
I got this from a "MultimediaWiki" website, and gstreamer also contains implementations (parsers instead of generators) of this in both gst-plugins-base-libs (pbutils/codec-utils.h) and gst-plugins-good (aacparse, rtpmp4apay, rtpmp4gpay). If it's still deemed not safe for inclusion in Wine, then maybe we can get away with just using a static, hardcoded user data, or maybe no user data at all. It's using Gstreamer for the AAC codec data now.
-- https://gitlab.winehq.org/wine/wine/-/merge_requests/5159#note_65501
On Thu Mar 21 05:05:06 2024 +0000, Torge Matthies wrote:
It's using Gstreamer for the AAC codec data now. Ech, this is... not great, sorry :-/
Documentation suggests that the builtin AAC encoder should have a MF_MT_USER_DATA in its media type. Is this wrong? Or is the application plugging its own AAC encoder? Or stripping that attribute? If we really need this I prefer to have the code in Wine manually, assuming it's deemed safe. -- https://gitlab.winehq.org/wine/wine/-/merge_requests/5159#note_65577
On Thu Mar 21 05:05:06 2024 +0000, Zebediah Figura wrote:
Ech, this is... not great, sorry :-/ Documentation suggests that the builtin AAC encoder should have a MF_MT_USER_DATA in its media type. Is this wrong? Or is the application plugging its own AAC encoder? Or stripping that attribute? If we really need this I prefer to have the code in Wine manually, assuming it's deemed safe. The AAC encoder does not require the `MF_MT_USER_DATA` attribute, if it's missing it will simply generate one from other attributes.
-- https://gitlab.winehq.org/wine/wine/-/merge_requests/5159#note_65609
On Thu Mar 21 10:57:13 2024 +0000, Torge Matthies wrote:
The AAC encoder does not require the `MF_MT_USER_DATA` attribute, if it's missing it will simply generate one from other attributes. I don't know how many things require the codec data from `mf_media_type_to_wg_format`, I know that at least the AAC encode IMFTransform and the muxer (IMFMediaSink) do not require the `MF_MT_USER_DATA` attribute, where the encoder generates it and makes it available in the output type, and the muxer has to generate it too because gstreamer complains if `codec_data` is missing from the caps.
-- https://gitlab.winehq.org/wine/wine/-/merge_requests/5159#note_65610
The AAC encoder does not require the `MF_MT_USER_DATA` attribute, if it's missing it will simply generate one from other attributes.
Sure, but that shouldn't require doing anything manually, right? We just need to query the codec_data that's set as caps on our sink pad. That'll require some code changes, but they'd be less awkward than reconstructing the codec_data from parsed data... Unless the native AAC encoder doesn't actually set that attribute? -- https://gitlab.winehq.org/wine/wine/-/merge_requests/5159#note_65692
On Thu Mar 21 19:31:14 2024 +0000, Zebediah Figura wrote:
The AAC encoder does not require the `MF_MT_USER_DATA` attribute, if it's missing it will simply generate one from other attributes. Sure, but that shouldn't require doing anything manually, right? We just need to query the codec_data that's set as caps on our sink pad. That'll require some code changes, but they'd be less awkward than reconstructing the codec_data from parsed data... Unless the native AAC encoder doesn't actually set that attribute? The AAC encoder actually doesn't expose the generated `MF_MT_USER_DATA` in the output type, you were right. Which only makes it more important that we either generate it in the IMFMediaSink or parse it from the input stream.
I already did an experiment with just putting an `aacparse` element at the start of the wg_muxer pipeline, but that didn't make GStreamer happy. It might be that the input to the media sink is an AAC stream in "raw" format, and I think you can't parse out codec data from that. In which case I don't see a way around generating the user data separately in some way from the attributes set on the input type of the IMFMediaSink, either directly in Wine or with the Gstreamer-based method. -- https://gitlab.winehq.org/wine/wine/-/merge_requests/5159#note_65975
On Mon Mar 25 09:34:51 2024 +0000, Torge Matthies wrote:
The AAC encoder actually doesn't expose the generated `MF_MT_USER_DATA` in the output type, you were right. Which only makes it more important that we either generate it in the IMFMediaSink or parse it from the input stream. I already did an experiment with just putting an `aacparse` element at the start of the wg_muxer pipeline, but that didn't make GStreamer happy. It might be that the input to the media sink is an AAC stream in "raw" format, and I think you can't parse out codec data from that. In which case I don't see a way around generating the user data separately in some way from the attributes set on the input type of the IMFMediaSink, either directly in Wine or with the Gstreamer-based method. Yes, reconstructing codec data is just not something aacparse does unfortunately. That could of course be changed but it would put an unfortunate limit on the minimum GStreamer version, and it's not all that clear that it's the right thing to do here.
-- https://gitlab.winehq.org/wine/wine/-/merge_requests/5159#note_66029
The AAC encoder actually doesn't expose the generated `MF_MT_USER_DATA` in the output type, you were right. Which only makes it more important that we either generate it in the IMFMediaSink or parse it from the input stream.
Interesting, the documentation claims otherwise. It may be worth adding a test that explicitly shows this. -- https://gitlab.winehq.org/wine/wine/-/merge_requests/5159#note_66030
participants (4)
-
Marvin -
Torge Matthies -
Torge Matthies (＠tmatthies)
-
Zebediah Figura (＠zfigura)