Hi,
I have tested your application directly(wine ./main) and it seems to work for me. (note that i can't compile it because i don't have "i486-linux-gnu-gcc") It switch to 640*480 and when i press a key (or left mouse button), all the screen changes to a color (to white for mouse except a little part in the bottom right that shows a tiny code bars). But when i use ESC key, i quit the application but doesn't back to my 1024*768 mode. I must restart my X server (crtl+alt+backspace) and re-login. As you say it appears not to work with open source driver, it works for me who use dri driver. If you have another tests or you need more information on my system, i can help you
Hm. Odd. What card do you have? Can you try with software rendering(take away the users write rights to /dev/dri/card* )
Anyway, the locking code needs some work, and some optimization.
Stefan