I tried the unlinking/linking stream decodebin approach. However, it's really complex to implement that, because changing pipeline dynamically needs to consider a lot of stuff (eg. renegotiate caps, seek to restart pipeline, flush previous buffers). Also, buffer loss happens because of the pipeline changes.
Instead, adding a probe is quite simple and easy to implement.
Using a probe means that the samples will get fed into the decodebin anyway, which ends up just wasting CPU on decoding.
We can prevent the buffer from going into stream decodebin by return GST_PAD_PROBE_DROP or GST_PAD_PROBE_HANDLED in probe, which I think is identical to "detach".
Hmm, okay, interesting. I had been assuming that all of this was unavoidable, but on reflection it does seem that the probe is surprisingly simple and should work.
After OpenStream, the decoding thread in our pipeline is already started. So before SetReadStreamSamples, there is posibility that a uncompressed frame, which is the first frame, is already in our stream->buffer. If we detach the stream decodebin in SetReadStreamSamples, the compressed buffer we grab in GetNextSample is the second frame, unless we seek to where we want to be.
So this is already a problem we have when changing the output format (between two uncompressed formats). At that point I figured that the problem is intractable, and that the sensible thing to do would just be to seek back to the beginning: https://source.winehq.org/git/wine.git/blob/HEAD:/dlls/winegstreamer/wm_reader.c#l2244
The idea of grabbing a reference to the compressed sample is interesting, and I feel like it *almost* solves the problem. The only issue is that the decoding pipeline isn't forbidden to have a queue in it somewhere, which would violate the assumptions we make here... but in practice it's not particularly likely to, so it could work anyway.
Of course, the other thing to consider is that switching from compressed to uncompressed mid-stream seems even less likely than switching formats, and if we decide we don't want to care about that either, we can probably simplify some logic—like not using separate fields for the compressed and uncompressed streams. But on the other hand, the patch as-is doesn't feel complex enough to bother me, at least now that I understand the idea.
One thing we may consider doing, that would probably simplify things a bit, is not mapping samples immediately. I think that's a holdover from when we would return the sample address directly to the client, but we're not doing that anymore—we could just map and unmap it in wg_parser_stream_copy_buffer(). As far as this patch is concerned that would remove the need to map the compressed and uncompressed samples separately, and remove a couple of ternaries.