Hi Rémi,
I don't know much about the input thing, but do you know what Windows does when you Alt-Tab? Does it send Alt release immediately to the window, or when you actually release it? (even without focus)
If it's the former, I guess one easy way would be to simply release all keys that are pressed when they keyboard is grabbed. Not sure if X11 has such a notification facility.
But I'm pretty sure you already looked into this and turned out it's more complicated than that. :-)