I think you seem to be under the impression that I've been saying we shouldn't output compressed streams from the MF media source, or wg_parser. I'm not, and if I've miscommunicated to the contrary, I apologize. Even back when the media source was first submitted, I recognized that we might need to do that eventually—I just said "let's start with decoding all the way to raw formats, and add specific compressed streams when we need to," and then it turned out we didn't need to for a long time.
To clarify statements I've made in the past, I have *resisted* actually doing that work, partly because it's a lot of work (not just for someone to do, but for us to have in tree), and partly because it makes our pipelines more complicated, harder to debug, and (at least in theory) less performant. In those cases where applications make assumptions I've seen it as fine to just incrementally tweak the way we output uncompressed streams, one part a time, because that usually works. I have a hard time calling these hacks, because they almost by definition don't go against the design of quartz (or mfplat), they just make our custom elements act a little more like some specific real element. I don't think "it's the way Windows does it internally" is a good enough reason to design something internally in Wine *if a program doesn't care*, and *if you have a good reason to do it differently*. Historically I think both of those have been the case.
Nevertheless, I've never tried to say we *shouldn't* expose compressed streams directly. And recently we have found applications that really do need compressed streams, in ways that are hard or impossible to avoid just by having our generic decoder act as a better simulacrum.
What I have been trying to ask is: why do we need to replace wg_parser? And, separately, why do we need to replace decodebin? And if decodebin has problems, can they be fixed on the GStreamer level instead of just giving up on those libraries?
Ziqing has a proposal for allowing decodebin, and hence wg_parser, to output compressed sample data, and I think it makes a lot of sense. However, you say that it's not going to work, or that decodebin is not a good fit for winegstreamer. I know you've done a lot of work on multimedia, so if you say that decodebin has problems, or that there are problems with Ziqing's approach, I want to take that into account.
One problem you mention with decodebin is that it exposes streams in a non-deterministic order. This makes sense, I think. However, I can't say that the idea of solving this by completely rewriting decodebin sits particularly well with me. The particular reason is that even with the new code, we're not really relying on documented, API-guaranteed behaviour. And I'm not dead-set against leaning on internals a bit, but I'm not thrilled about it either, and I don't like the idea of introducing a large amount of code just so that we can lean on *different* internals. Point is, if we need streams to be ordered in some specific way—the order in which they're enumerated in the file, for instance—can we get GStreamer to provide this to us explicitly, through some tag? But even if that's a non-starter, and we have to lean on internals, can we lean on the order in which our autoplug-continue signals are sent, maybe?
Another problem I've gathered is that either decodebin or wg_parser is too slow to initialize streams, and this trips up an Unreal Engine bug. What I haven't fully gathered, though, is what part(s) of the process are slow. Is it decodebin autoplugging? Our prerolling? Something else? If the problem is decodebin autoplugging, can we resolve that by returning as soon as we have all of the uncompressed streams from autoplug-continue? If it's our prerolling, can we try to return earlier from wg_parser_connect(), and defer the rest of the prerolling process to when we really need it?
And, if in the end it turns out that we really can't use decodebin, why do we need to replace wg_parser? Why can't we just replace the innards of wg_parser, and leave the interface and frontends untouched?