On Mon Sep 25 16:06:41 2023 +0000, Rémi Bernon wrote:
I'm not sure, but I think maybe it could handle the format change from the caller side? Calling `wg_transform_set_output_format` with the new format if the change is no-op from the quartz PoV? For reference, I've stripped the default values for these properties on input, in 59dedf2754650bd84ffa7d10c94f83b97198fed2, to let decoders decide what their values should be and avoid forcing a colorspace conversion in videoconvert. Then we might be interested to expose them to the client side at some point, MF has some attributes to represent colorimetry information, but I don't know if AM_MEDIA_TYPE has something for it as well.
This function's immediate caller is GStreamer itself; we can't change that. It may be possible to push the 'is it actually different' check later, but not earlier. And that 'later' part looks full of multithreading already, I'd rather not make it even trickier.
Another approach would be adding a colorimetry=(string)2:0:0:0 somewhere in wg_transform_create(); if it's already there, avdec_mpeg2video adding one won't do anything.
But the clean way to do that would be in wg_format_to_caps(), which is impossible. The wg_format is created from an AM_MEDIA_TYPE, which most likely contains a VIDEOINFOHEADER, which does not support colorimetry. AM_MEDIA_TYPE can contain VIDEOINFOHEADER2 that contains some color-related fields in the DXVA_ExtendedFormat field in the control flags, but I can't quite make sense of which (if any) of those bits is colorimetry, and we can't rely on in-the-wild programs accepting VIH2 anyways.
The less clean way would be hardcoding a 'if input format is WG_MAJOR_TYPE_VIDEO_MPEG1, add colorimetry 2:0:0:0 to output format' somewhere, but that's at least as ugly as the current approach. And if some other MPEG file contains a colorimetry other than 2:0:0:0 (I don't know if that's possible), we're gonna get unexpected format changes again.
The ideal solution would be converting all queued frames to the new format (or, if some frames have already been sent, queueing the format change too), so format changes don't cause trouble, but that's a bit beyond the scope of this MR.
Since there already is some colorimetry removal in winegstreamer (for example that commit you linked), I'd say removing it from here too is the cleanest general solution (unless I'm missing some other possible solution). Though it may be wiser to refactor that caps filtering into a separate function.