Turns out I broke the non virtual desktop case in some cases. Sorry about that.
This basically uses previous behavior for such (`enable_taskbar` is what's used to also determine layered changes in rest of code) while keeping the fix for virtual desktops (i.e. our own taskbar).
Works correctly for me, but let me know if I did something wrong again (welp).