What version did you specify in ~/.wine/config [windows] version?
win2k
I guess you ran in non NT mode. And I guess your change will do havoc to programs needing NT mode (like Xilix ISE). Probably the limit has to be set in dependance of the version emulated.
The limit doesn't depend on version, at least not in terms of what you probably mean by version. Windows only offers a 3GB user address space on its "higher end" editions, such as Advanced Server. Some applications can't deal with this. Yes, technically this is a bug in the application, but it's a bug that would not be triggered when running on a genuine Microsoft workstation operating system.
And yes, it should be configurable, or perhaps Wine should be capable of detecting which apps can deal with a 3GB user address space (as apparently executables can be marked to declare this), but as long as it's not configurable, and Wine's not capable of detecting this, it seems to me that the 2GB limit applicable to the desktop operating systems is the correct one.