It confuses SDL which automatically restores window when it is being activated, and changes display mode on fullscreen window restore.
The extra window restore will try to restore the window again to its normal rectangle, but this time translated to the new display mode, ending up with a larger than monitor window resize. This confuses SDL, which then decides to restore the native display mode.