On Fri Jul 5 01:31:36 2024 +0000, Rémi Bernon wrote:
I don't think we should do any copy at all here. When the source reader is D3D-aware, and when its pipeline contains either a decoder or a video processor (which is normally the case 99.9% of the time, and not the case only when playing raw video), they will allocate D3D samples already and we don't need to copy anything.
I'm not super familiar with the `IMFSourceReader`, so I might have some of this wrong. But it looks like the current design receives samples from the `IMFMediaStream` asynchronously and then queues them.
The copy takes place when the application calls `IMFSourceReader::ReadSample` and the application is provided a sample. The sample received by the application is created by an internally managed `IMFVideoSampleAllocatorEx`, whilst the samples from `IMFMediaStream` are created by the media stream itself.
So I think under this design, a copy is unavoidable. Is it designed that way to match the Windows implementation? Or do we have scope for change?
Also of note, it also looks like `IMFMediaStream` is not currently D3D-aware. I think we would need to implement `IMFMediaSourceEx` so we can call `IMFMediaSourceEx::SetD3DManager`. I guess we will want to do this if we want to receive YUV output in a D3D texture for potential color conversion by the GPU.