What does "windows firefox thinks it's in fullscreen mode" mean? Does the application try to manually resize itself in addition to sending the state change messages to the root window (perhaps as a workaround for WMs not supporting the _NET_WM_STATE stuff)? Or is there some other notification that the window is attempting to be in fullscreen mode?
Windows does not have a fullscreen mode. In Windows, windows that are Always on Top are on top of panels. Windows applications enter "fullscreen mode" by positioning themselves take up the whole screen and being always on top (actually, now I'm not so sure they have to be always on top; I'm only basing that on my experience as a Windows user).
When the size of a window or the screen resolution changes, Wine marks windows whose client area covers the whole screen as fullscreen. This is done in the update_fullscreen_state function in this file:
http://cvs.winehq.org/cvsweb/wine/dlls/winex11.drv/winpos.c?rev=1.5&cont...
How are you making firefox try to enter fullscreen mode? (F11?)
Yep.
You see part of the bottom panel, but not the whole thing and you can't interact with any of it? Sounds like an application bug where Metacity has resized and placed the window on top but the app just hasn't updated from expose events. You could verify by e.g. running sleep 5; xprop | less in a terminal, switching to firefox during the sleep and putting it in fullscreen mode, then after the sleep clicking in the area where you see part of the bottom panel. xprop lists all kinds of crap, but the WM_CLASS or WM_NAME (both near the end) are usually pretty good ways to verify which application you clicked on.
That issue is with Alt+F11 (metacity forcing fullscreen mode), and it's not really as important, since Windows can't do anything like that.
Alternatively, I can point you to the source code in metacity. The Alt-f11 keybinding I had you set up is handled in metacity/src/keybindings.c:handle_toggle_fullscreen(). Messages from apps to add/remove/toggle these state properties are handled in metacity/src/window.c:meta_window_client_message() you'll want to search for "fullscreen" within that function as many other states are handled in that function as well.
I'll look at that when I have a chance (even though I'm not really an expert on this stuff), thanks.