On Thu Feb 23 11:25:41 2023 +0000, Tingzhong Luo wrote:
Sorry that I have not found any code to reset `mixer->output.rt_formats` or `mixer->output.rt_formats_count` in current source tree. It might cause another crash if application want to clear media type of mixer twice, like this:
hr = IMFTransform_SetInputType(transform, 0, NULL, 0); hr = IMFTransform_SetInputType(transform, 0, NULL, 0);
in this case, it `mixer->output.rt_formats` has been free, but it try to release a invalid pointer in second call. Another way to cause a crash will looks like this:
hr = IMFTransform_SetInputType(transform, 0, NULL, 0); hr = IMFTransform_SetOutputType(transform, 0, media_type, 0);
`output.rt_formats[i].media_type` might be released and IMFTransform_SetOutputType() try to access it. In v3 of this patch, those crash can be reproduce in the test if reverse changes in `video_mixer_clear_types()`.
For that we could memset() whole mixer->output structure in video_mixer_clear_types().