On Wed Apr 3 12:15:37 2024 +0000, Rémi Bernon wrote:
Yes, this is a big difference in the way the media source is implemented, and it causes all sorts of issues in other places. I've been slowly working toward changing it but it requires a lot of changes in the MF pipeline to support it. Many applications implicitly or explicitly depend on MF pipelines exposing similar components as on Windows (source -> decoder -> converter -> sink), and we should implement the same kind of thing for compatibility. If the decoders are taking a long time to decode and stall each-other, it should be solved by making them as concurrent as they are on Windows. There's also some diverging opinions on how the media source itself should be implemented, where I would like to use a separate unix interface to make it simpler and synchronous, while @zfigura doesn't want that.
To add a bit more detail about the issue, the two main causes for the 40ms wait appear to be: 1. The decoding that takes place in the Theora decoder; and 2. The memcpy that takes place in wg_parser_stream_copy_buffer
I've tried a hack to remove the memcpy, but it's unfortunately not enough; and the decoding is obviously unavoidable. So the only fix appears to be to have the video and audio happen in parallel (not sequentially).
Obviously having the video decode happen in a separate MFT (in its own thread) would do the job, but do you think there's a place for this MR in the interim? Or do you think the additional concurrency it introduces is too great a risk?