Hello Keming,
It isn't clear from the log, but it looks like the application may be trying to create a second filter on the same device without fully releasing the first one.
Although I haven't tested very fully, I think Windows is more lenient about opening multiple capture filters than we currently allow for. It may be that there's a leak somewhere that we're to blame for; it's probably worth checking carefully to make sure that isn't the case.
But we may need to find ways to avoid holding a file descriptor open when not streaming. When doing that you will need to account for the fact that most APIs remain usable even after a stream is stopped. You may want to defer VIDIOC_S_FMT until streaming starts, and find a way to release ownership of the device when streaming stops (I don't know offhand if v4l2 provides this; you may need to close and reopen the descriptor.)
Tests replicating the application behaviour would also be useful.
ἔρρωσο, Zebediah