On Saturday 19 April 2003 10:40 am, Ove Kaaven wrote:
I see that Greg's exception stuff is not in Wine yet. Until that's accepted, this'll have to do.
hmm, Alexandre, perhaps I should try to produce an acceptable version of those exception-handling macros which get activated iff gcc is the compiler (and otherwise we could revert to the dummy versions)?
"Gregory M. Turner" gmturner007@ameritech.net writes:
hmm, Alexandre, perhaps I should try to produce an acceptable version of those exception-handling macros which get activated iff gcc is the compiler (and otherwise we could revert to the dummy versions)?
The problem with that is that the code will then start to depend on exceptions, since they will work for 99% of the people using Wine, and they will never get fixed the proper way. I'd really prefer that we find a portable solution.
On April 21, 2003 08:19 pm, Alexandre Julliard wrote:
The problem with that is that the code will then start to depend on exceptions, since they will work for 99% of the people using Wine, and they will never get fixed the proper way. I'd really prefer that we find a portable solution.
Just like with threads, why not cross that bridge when we get there? Is it really worth it to penalize 99% of the people? When we port to another compiler, it will be a lot simpler to worry about those pieces of code, rather than have no code at all.
Also, not accepting code into Wine that depend on exception is one thing, but what about Winelib? We can have the exception code commented out if __WINESRC__ is defined, but at least lets get it in if it works, it will plug a big Winelib hole.
"Dimitrie O. Paun" dpaun@rogers.com writes:
Just like with threads, why not cross that bridge when we get there? Is it really worth it to penalize 99% of the people? When we port to another compiler, it will be a lot simpler to worry about those pieces of code, rather than have no code at all.
The difference with threads is that we know all platforms have some kind of threading, so we know that we can adapt the code when needed. I don't think any other C compiler supports local functions (even g++ doesn't AFAIK), so there won't be much hope of ever making it work. Of course we could decide to stop supporting other compilers, but I don't think everybody would be happy with that.
Also, not accepting code into Wine that depend on exception is one thing, but what about Winelib? We can have the exception code commented out if __WINESRC__ is defined, but at least lets get it in if it works, it will plug a big Winelib hole.
Sure, we can make the macros available for Winelib.
On April 21, 2003 11:55 pm, Alexandre Julliard wrote:
I don't think any other C compiler supports local functions (even g++ doesn't AFAIK), so there won't be much hope of ever making it work. Of course we could decide to stop supporting other compilers, but I don't think everybody would be happy with that.
True. But we should also ask what benefit we get from those other compilers? BTW, what other compilers do people use that don't support exceptions? It would be interesting to compile Wine with MS' cl, or Borland's bcc. But both support exceptions. The only other compiler would be Sun's cc, but Patrik has given up on that one a long time ago: http://www.dssd.ca/wine/Wine-Fun.html#source
Also, it seems to me that the benefit of compiling the code with other compilers is to get some warnings/errors not generated by gcc, but that can be achieved just fine if we stub out the exception handling code. Yeah, we may not get a running Wine, but so what?
Moreover, supporting only gcc for the moment (and look at the kernel, it takes a long time to find a reasonable alternative, we didn't and we are 10 years old!) would allow us to use winegcc internally, which would bring us other benefits.
So yeah, I think we should drop support for generating a running executable for compilers that don't have a certain level of features. If people disagree with that, let them step forward with convincing cost/benefit ratios and working patches.
"Dimitrie O. Paun" dpaun@rogers.com writes:
True. But we should also ask what benefit we get from those other compilers? BTW, what other compilers do people use that don't support exceptions? It would be interesting to compile Wine with MS' cl, or Borland's bcc. But both support exceptions. The only other compiler would be Sun's cc, but Patrik has given up on that one a long time ago:
AFAIK Sun's cc works pretty well, Gregg Mattinson did a lot of work to fix it; not sure if it's 100% but it should be close.
Also, it seems to me that the benefit of compiling the code with other compilers is to get some warnings/errors not generated by gcc, but that can be achieved just fine if we stub out the exception handling code. Yeah, we may not get a running Wine, but so what?
I don't think we can reasonably require people to make the effort to write portable code and avoid gcc-isms, while knowing that no other Unix compiler is ever going to work right anyway. There are a number of advantages to being portable, there would also be a number of different advantages to being gcc-specific, but being somewhere in between is not worth the trouble IMO.
On 22 Apr 2003, Alexandre Julliard wrote:
Unix compiler is ever going to work right anyway. There are a number of advantages to being portable, there would also be a number of different advantages to being gcc-specific, but being somewhere in between is not worth the trouble IMO.
Well, I certainly agree that there are advantages from being portable. That's why I have this sort of portability listed on the Fun page. But I also argue that it should not be detrimental to new features. Supporting Unix compilers other than gcc is a very, very niche thing. Let's not delay important and very useful features for such esoteric benefits. Once we have the code in and tested, it is so much easier to work around whatever limitations you have, if the will to do so it's there.
If this means that we'll be gcc-specific (on Unix, at least), I say let's go ahead and be gcc-specific on Unix. We have already too many things to worry about, any help is welcomed. Linux went this way, and it seems to be working fine. I'm sure that at the end of the day everybody will be much happier with a featureful Wine that compiles only on gcc, rather than a crippled but super compiler-portable Wine.
"Dimitrie O. Paun" dimi@intelliware.ca writes:
Well, I certainly agree that there are advantages from being portable. That's why I have this sort of portability listed on the Fun page. But I also argue that it should not be detrimental to new features.
It's not really new features, we already support exception handling, and it should work with all compilers. The only thing that doesn't work is using the Microsoft syntax, but that isn't such a big deal since that code will ultimately be generated by widl anyway, so we can make it generate it the way we want.
So I think exception handling by itself is not a sufficient reason for giving up on portability; but clearly, there are other arguments for being gcc-specific too, and the sum of these arguments does make a more convincing case. In fact I think the argument about winegcc and how it would allow simplifying the build process is more convincing than the exception macros.
What do the Solaris folks think? Would having to use gcc be a problem for you?
On Monday 21 April 2003 10:55 pm, Alexandre Julliard wrote:
"Dimitrie O. Paun" dpaun@rogers.com writes: needed. I don't think any other C compiler supports local functions (even g++ doesn't AFAIK), so there won't be much hope of ever making it work.
gosh, that sucks.
Of course we could decide to stop supporting other compilers, but I don't think everybody would be happy with that.
me neither. I guess this is kind of crazy, since they still aren't portable, but my goal in writing that code was to violate only one rule: no nested functions. of course, when I eventually used __label__ and "auto" I think I broke even those rules, making it totally gcc specific.
also,
Perhaps you will all emit a collective groan (or simply pity me for my inability to learn from history), but given __LINE__, and the usual macro-mashing-together tricks, I see no reason to assume that there is insufficient expressive power to achieve the proper semantics... I suspect a failure of imagination on my part. I'd like to take another look at this with a mind to reimplement all flow-of-control using goto instead of loops and nested functions.
I would hate to see wine scrap such a long-standing tradition as code portability just on account of some exception handling macros that are grossly inefficient wrappers anyhow...
Also, not accepting code into Wine that depend on exception is one thing, but what about Winelib? We can have the exception code commented out if __WINESRC__ is defined, but at least lets get it in if it works, it will plug a big Winelib hole.
Sure, we can make the macros available for Winelib.
I hate to be the weak link in the chain here... But I'd at least like to take this to the drawing board once more before I throw in the towel. I have some time on Thursday, so I will get back to the list on Friday with either a patch or a white flag.
On Monday 21 April 2003 10:55 pm, Alexandre Julliard wrote:
"Dimitrie O. Paun" dpaun@rogers.com writes:
Just like with threads, why not cross that bridge when we get there? Is it really worth it to penalize 99% of the people? When we port to another compiler, it will be a lot simpler to worry about those pieces of code, rather than have no code at all.
The difference with threads is that we know all platforms have some kind of threading, so we know that we can adapt the code when needed. I don't think any other C compiler supports local functions (even g++ doesn't AFAIK), so there won't be much hope of ever making it work. Of course we could decide to stop supporting other compilers, but I don't think everybody would be happy with that.
Also, not accepting code into Wine that depend on exception is one thing, but what about Winelib? We can have the exception code commented out if __WINESRC__ is defined, but at least lets get it in if it works, it will plug a big Winelib hole.
Sure, we can make the macros available for Winelib.
OK, folks, I think I may have a plan to create a portable __except, please let me know your thoughts, as there are many places I might be going wrong. What follows is a totally new approach compared to my old attempts; I haven't tried to implement a similar implementation "plan" for __finally yet.
Allright, here goes, in crap / gibberish ("pseudocode") form:
/* GLOBAL structure instance somewhere (ntdll?) Thread-local */ typedef struct { bool do_handler_bit; DWORD exception_code; jmp_buf jmp; } SEH_TLS_TEMP;
SEH_TLS_TEMP SEH_TLS_TEMPS;
WINE_EXCEPTION_HANDLER_FUNC WineSEHHandler(args) { if (setjmp(SEH_TLS_TEMPS.jmp)) { switch(SEH_TLS_TEMPS.exception_code) ... /* the rest is like __wine_exception_handler in ntdll/exception.c */ } else { do_handler_bit = 0; longjmp(exception_record_argument->jmpbuf, 0); } }
/* now here are the parts that would be in include/wine/exception.h */
/* exactly the same as __TRY, really just #define __try __TRY */ #define __try \ do { __WINE_FRAME __f; \ int __first = 1; \ for (;;) if (!__first) \ { \ do {
#define __except(expr) \ } while(0); \ SEH_TLS_TEMPS.do_handler_bit = 0; __wine_pop_frame( &__f.frame ); \ break; \ } else { \ __f.frame.Handler = (PEXCEPTION_HANDLER) WineSEHHandler; \ __f.u.filter = NULL; \ __wine_push_frame( &__f.frame ); \ if (setjmp( __f.jmp)) { \ int ecode; \ const __WINE_FRAME * const __eptr WINE_UNUSED = &__f; \ /* exception code hacks go here... */ \ if ((ecode = (expr)) != EXECUTE_HANDLER) { \ SEH_TLS_TEMPS.exception_code = ecode \ longjmp(SEH_TLS_TEMPS.jmp, 0); \ } \ RtlUnwind(...); \ __wine_pop_frame(); SEH_TLS_TEMPS.do_handler_bit = 1; \ break; \ } \ __first = 0; \ } \ } while (0); \ if (SEH_TLS_TEMPS.do_handler_bit)
This obviously is leaving some stuff out. For example, this wouldn't work with nested exceptions. However, that seems possible to fix, if not by abusing the jmp_buf right in the frame structure then by creating a stack of SEH_TLS_TEMP structures instead of just a single instance, or maybe in some other clever way like attaching the SEH_TLS_TEMP struct to the wine frame struct.
What I'm more concerned about is the following:
A() runs setjmp, calls B(), which calls C(), which calls D(). A() would be the user's function (whatever it is) and D() would be WineSEHHandler above. D() runs setjmp again on a second jump buffer, and calls longjmp to do a nonlocal goto into A().
Now:
(a) if the exception code runs, A() keeps running and eventually just returns... will it properly release all the stack from B(), C() and so on or is this a problem? I guess its OK since the SP just goes back to what it was when I called setjmp, right?
(b) if the exception code doesn't run, A() runs longjmp again, to non-local goto /back/ into D(), using the second jmp_buf. This returns normally and would, I guess, expect to unwind the stack normally through C(), B() and so on... (this is all ignoring the fact that flow-of-control was interrupted by an exception, but for my purposes I don't think it matters).
Are either of these scenarios illegal (in wine)? If not, I think I can make it work. Another concern would be that two longjmps are just too slow to be used, but since that only occurs in an exception scenario I think it's OK...
Thoughts, flames, etc?
tor, 01.05.2003 kl. 07.26 skrev Gregory M. Turner:
(a) if the exception code runs, A() keeps running and eventually just returns... will it properly release all the stack from B(), C() and so on or is this a problem? I guess its OK since the SP just goes back to what it was when I called setjmp, right?
This would be OK.
(b) if the exception code doesn't run, A() runs longjmp again, to non-local goto /back/ into D(), using the second jmp_buf. This returns normally and would, I guess, expect to unwind the stack normally through C(), B() and so on... (this is all ignoring the fact that flow-of-control was interrupted by an exception, but for my purposes I don't think it matters).
This is not OK. Since the first setjmp releases all the stack frames between A and D, trying to later reclaim that released stack with another setjmp is doomed to unpredictable stack corruption.
On Thursday 01 May 2003 01:42 am, Ove Kaaven wrote:
tor, 01.05.2003 kl. 07.26 skrev Gregory M. Turner:
(b) if the exception code doesn't run, A() runs longjmp again, to non-local goto /back/ into D(), using the second jmp_buf. This returns normally and would, I guess, expect to unwind the stack normally through C(), B() and so on... (this is all ignoring the fact that flow-of-control was interrupted by an exception, but for my purposes I don't think it matters).
This is not OK. Since the first setjmp releases all the stack frames between A and D, trying to later reclaim that released stack with another setjmp is doomed to unpredictable stack corruption.
was afraid of that... bummer. well, back to the drawing board ;) I guess this is how we ended up with the need for a function in the first place -- you (seem to) need "lambda" delayed execution capabilities that aren't easily achieved by a macro... :(
Say, Alexandre, would it be a possibility to just bake some magic into wpp for this?
On May 1, 2003 02:58 am, Gregory M. Turner wrote:
Say, Alexandre, would it be a possibility to just bake some magic into wpp for this?
A gcc-specific solution would be preferable IMO.
"Gregory M. Turner" gmturner007@ameritech.net writes:
Say, Alexandre, would it be a possibility to just bake some magic into wpp for this?
I don't see any reasonable way of doing that, you really need some compiler support. Note that even with gcc nested functions it won't be 100% correct, for instance a return inside a try block will corrupt the handler list; the only way to fully support exceptions would be to make gcc aware of them.
On Thursday 01 May 2003 12:25 pm, Alexandre Julliard wrote:
"Gregory M. Turner" gmturner007@ameritech.net writes:
Say, Alexandre, would it be a possibility to just bake some magic into wpp for this?
I don't see any reasonable way of doing that, you really need some compiler support. Note that even with gcc nested functions it won't be 100% correct, for instance a return inside a try block will corrupt the handler list; the only way to fully support exceptions would be to make gcc aware of them.
hmm, you say this on the assumption that the try block would be a nested function itself? Because I think I could work a gcc-specific version so that there really is no nested function except perhaps for the "expr" evaluator, as in
__try { } __except ( expr ) { }.
Basically, except for the additional hassle of non-portability it should be no better or worse than __TRY... (which does have some documented limitations, but seems pretty darn close to what we want).
Or, do you say this because there would be no __wine_pop_frame if they returned? Couldn't we overwrite the return address pointer on the stack and then (if & when it is used) jump to the real return address after neccesary clean-up's...? Does __FINALLY suffer from this problem?
I dunno... probably you are right... even if there is a way, it sure is a hell of a brainteaser.
"Gregory M. Turner" gmturner007@ameritech.net writes:
Or, do you say this because there would be no __wine_pop_frame if they returned? Couldn't we overwrite the return address pointer on the stack and then (if & when it is used) jump to the real return address after neccesary clean-up's...? Does __FINALLY suffer from this problem?
Yes, of course the existing macros have the same problem; the thing is that with the current macros you need to fix the code anyway, so you can fix the other problems too and make sure you work within the limitations of the macros. The problem with having more compatible macros is that the code will compile just fine, but in many cases will break at run time, and you'd need to go through all the code manually to find the problematic spots.
tor, 01.05.2003 kl. 19.25 skrev Alexandre Julliard:
"Gregory M. Turner" gmturner007@ameritech.net writes:
Say, Alexandre, would it be a possibility to just bake some magic into wpp for this?
I don't see any reasonable way of doing that, you really need some compiler support. Note that even with gcc nested functions it won't be 100% correct, for instance a return inside a try block will corrupt the handler list;
Hmm. That could be solved by making the try block a nested function too, but I suppose that would bring some overhead...
On May 5, 2003 12:18 am, Ove Kaaven wrote:
Hmm. That could be solved by making the try block a nested function too, but I suppose that would bring some overhead...
If it fixes the problem, I'd take the overhead any time. If people are concerned about that, they can: A. Use a compiler that support the sytanx B. Rewrite the code to avoid using our compatibility macros
Ove Kaaven ovek@arcticnet.no writes:
Hmm. That could be solved by making the try block a nested function too, but I suppose that would bring some overhead...
Even ignoring the overhead, I can't think of a way to declare that function with the right return type. But maybe I missed something...
tir, 06.05.2003 kl. 01.01 skrev Alexandre Julliard:
Ove Kaaven ovek@arcticnet.no writes:
Hmm. That could be solved by making the try block a nested function too, but I suppose that would bring some overhead...
Even ignoring the overhead, I can't think of a way to declare that function with the right return type. But maybe I missed something...
Hmm, true. I thought it might be possible to do some magic like typeof(func()), but then you'd need to know the name of the function, and I can't seem to find a gcc builtin that'll retrieve a reference to the current function (or its return type). Anyone else have any ideas? (No, __FUNCTION__ is a string, so it's no good.)
On Monday 05 May 2003 07:28 pm, Ove Kaaven wrote:
tir, 06.05.2003 kl. 01.01 skrev Alexandre Julliard:
Ove Kaaven ovek@arcticnet.no writes:
Hmm. That could be solved by making the try block a nested function too, but I suppose that would bring some overhead...
Even ignoring the overhead, I can't think of a way to declare that function with the right return type. But maybe I missed something...
Hmm, true. I thought it might be possible to do some magic like typeof(func()), but then you'd need to know the name of the function, and I can't seem to find a gcc builtin that'll retrieve a reference to the current function (or its return type). Anyone else have any ideas? (No, __FUNCTION__ is a string, so it's no good.)
hmm, that sucks... there are some fancy __builtin_apply and __builtin_return thingys in gcc, not sure if those would help or not, they look like a bit of a research project in-and-of-themselves.
Of course, by using nested functions, this trick would limit our implementation to gcc, no g++. I figured out how VC++ does it: return does, indeed, become a fancy unwind/jump sequence.
The more I think about this one the more discouraged I become... My idea of stealing the return adress on the stack is also no good: by the time our routine gets called, all the stack objects the __try {} block wants to refer to are gone.
There might be some really clever trick involving manually creating simulated stack frames, but I think it would require looking at data on the stack which is deallocated... FWIW, it appears that vc++ does engage in some cheating of a similar nature, violating the "Thou shalt not look beyond the top of the stack" commandment.
btw ReactOS has a gcc patch for this:
http://reactos.wox.org/index.php?page=gccseh
they document several limitations there, so they aren't 100% done either, and obviously, requiring compiler patches to build wine isn't going to improve portability ;)
I think we have about three choices:
1) implement some really, really, really agressive stack mangling, and stick with macros somehow.
2) implement a parser/generator of some kind to do it ourselves
3) focus on convincing the gcc dev's to put it in, and only use the feature in wine if it's built into the compiler.
I don't like #3 and implementing #1 is looking pretty grim.
#1: so far nobody has thought of a way to do it; it's a dream, not a plan. The possibilities we have explored, so far, would fail in g++ due to nested functions, and break various other language features they aren't supposed to.
#3: not portable, and therefore not generally usable in wine.
That leaves #2. #2 could solve it for wine, ReactOS, anybody who wants it, really. There are obviously certain hassles with another layer of parsing, like slower compiles, making sure we don't break debugging, actually implementing it, etc.
But I'm beginning to believe Alexandre that #1 is impossible (or at least beyond the capabilities of my imagination) without more expressive power at the language level.
tir, 06.05.2003 kl. 20.23 skrev Gregory M. Turner:
On Monday 05 May 2003 07:28 pm, Ove Kaaven wrote:
tir, 06.05.2003 kl. 01.01 skrev Alexandre Julliard:
Ove Kaaven ovek@arcticnet.no writes:
Hmm. That could be solved by making the try block a nested function too, but I suppose that would bring some overhead...
Even ignoring the overhead, I can't think of a way to declare that function with the right return type. But maybe I missed something...
Hmm, true. I thought it might be possible to do some magic like typeof(func()), but then you'd need to know the name of the function, and I can't seem to find a gcc builtin that'll retrieve a reference to the current function (or its return type). Anyone else have any ideas? (No, __FUNCTION__ is a string, so it's no good.)
hmm, that sucks... there are some fancy __builtin_apply and __builtin_return thingys in gcc, not sure if those would help or not, they look like a bit of a research project in-and-of-themselves.
Of course, by using nested functions, this trick would limit our implementation to gcc, no g++.
Maybe this particular implementation, but C++ has other mechanisms that can be exploited. It might even be possible to bolt it on C++ native exceptions. If not, return could be caught by e.g. using a temporary object created in the try block which automatically unwinds the stack when it's destroyed unexpectedly. C++ is so rich that it shouldn't be hard to come up with such a technique, which is why I mostly concern myself with C-specific techniques right now, and don't care about they not working with g++; it should be completely acceptable to have one implementation if __cplusplus is defined and another if not, as long as it works, we don't have to limit ourselves to one implementation for both if it's impossible to have one that way.
Apart from the ugliness, what was wrong with hijacking the return address? It might be possible to put into the TEB any data you need that you expect will go out of scope by the time the hijacker-function gets invoked, I think.
On Tuesday 06 May 2003 06:50 pm, Ove Kaaven wrote:
Maybe this particular implementation, but C++ has other mechanisms that can be exploited. It might even be possible to bolt it on C++ native exceptions.
good point. For example, if we create a local object with a destructor that can call finally, we just implemented magical return-notification, since the language is guaranteed to call our destructor during unwinds due to an exception or a return... of course, the devil is most assuredly in the details ;)
Apart from the ugliness, what was wrong with hijacking the return address? It might be possible to put into the TEB any data you need that you expect will go out of scope by the time the hijacker-function gets invoked, I think.
Well, we could just memorize the stack between the frame pointer and the stack pointer, for example, and just put things back the way they were at some prior time,... this "remembering" trick would have to go at the beginning of the try block, however, which puts us in pretty dicey territory, a lot could change between the beginning of the try block and the exception, so now our frame is out-of-date, and the expected results are not achieved, i.e., local integers time-warp back to what they were at the beginning of the try block before the finally executes.
The only solution along this path that occurs to me is a frightening one: we assume (actually, we know, since this will only happen due to a return in the try block), upon invocation of our "interceptor" code, what happened: we were exeucting the try block, and the bastard returned. So we just act on the assumption that the stack frame is still sitting there, intact; we immediately restore frame-pointer, stack-pointer, registers, etc., say, from a setjmp() (not sure longjmp would do quite the right thing here, but maybe...), and branch to the finally clause or some perversion thereof. Then, when finally's done, we need to tear down everything back to what it was before we intercepted. (setjmp/longjmp should be fine for this second branch, into a secondary(?) JMP_BUF in the TEB), and continue on our merry way by magically returning the correct value to the caller... easy ;)
Seriously, I dunno, sounds pretty darn sketchy... we would need to be able to rest assured that the act of returning will not somehow mangle the stack on the way out, and that doesn't seem so clear to me...
Worth a shot, I guess....
-
Alexandre Julliard -
Alexandre Julliard -
Dimitrie O. Paun -
Dimitrie O. Paun -
Gregory M. Turner -
Ove Kaaven