Hi list,
I am currently having some problems with a change I am working on, that result in heap corruption as a result of buffer overrun. I was wondering whether there was any builtin debug options to detect and help fix these things.
If there aren't, I am perfectly willing to sidetrack a little and write them. Please let me know.
Shachar
On Sun, Sep 01, 2002 at 12:17:38PM +0300, Shachar Shemesh wrote:
Hi list,
I am currently having some problems with a change I am working on, that result in heap corruption as a result of buffer overrun. I was wondering whether there was any builtin debug options to detect and help fix these things.
If there aren't, I am perfectly willing to sidetrack a little and write them. Please let me know.
If you run with -debugmsg +heap it will check the heaps at every access for corruption.
Ciao, Marcus
Marcus Meissner wrote:
On Sun, Sep 01, 2002 at 12:17:38PM +0300, Shachar Shemesh wrote:
Hi list,
I am currently having some problems with a change I am working on, that result in heap corruption as a result of buffer overrun. I was wondering whether there was any builtin debug options to detect and help fix these things.
If there aren't, I am perfectly willing to sidetrack a little and write them. Please let me know.
If you run with -debugmsg +heap it will check the heaps at every access for corruption.
Ciao, Marcus
Thanks, but this is not what I meant.
Currently, when a buffer overruns, the free pointers right after get corrupted, and when the next piece of block is allocated, the machine will crash. This will tell me approximetly where the buffer overrun, but is, generally speaking, too late. Enabling the debug messages detects the error at the same place (i.e. - too late).
What I am talking about is a compile time (or run time, no reason why not) option that will add "hot zone" areas before and after each heap allocated buffer. These zones should be: A. Large enough so that the overrun doesn't reach the heap managment structures themselves. This means that the program can keep on running despite having the buffer overflow. B. Identifiable, so that we know if a buffer has overwritten it.
Whenever you free a buffer, you check the hot zones. If they have changed, the buffer you are freeing now has been overwritten. This gives an excellent debugging tool, as it points right at the offending code.
I am going to try and find my overrun buffer the conventional way. If that doesn't work, I'll implement the system described above. Who knows how many overflows are out there that do not crash the system, and go about undetected?
Shachar
On Sun, Sep 01, 2002 at 11:33:55PM +0300, Shachar Shemesh wrote:
Marcus Meissner wrote:
On Sun, Sep 01, 2002 at 12:17:38PM +0300, Shachar Shemesh wrote:
Hi list,
I am currently having some problems with a change I am working on, that result in heap corruption as a result of buffer overrun. I was wondering whether there was any builtin debug options to detect and help fix these things.
If there aren't, I am perfectly willing to sidetrack a little and write them. Please let me know.
If you run with -debugmsg +heap it will check the heaps at every access for corruption.
Ciao, Marcus
Thanks, but this is not what I meant.
Currently, when a buffer overruns, the free pointers right after get corrupted, and when the next piece of block is allocated, the machine will crash. This will tell me approximetly where the buffer overrun, but is, generally speaking, too late. Enabling the debug messages detects the error at the same place (i.e. - too late).
What I am talking about is a compile time (or run time, no reason why not) option that will add "hot zone" areas before and after each heap allocated buffer. These zones should be: A. Large enough so that the overrun doesn't reach the heap managment structures themselves. This means that the program can keep on running despite having the buffer overflow. B. Identifiable, so that we know if a buffer has overwritten it.
No, as Marcus says, enabling +heap debugging forces the entire heap to be checked for consistency on every HeapAlloc/Free/ReAlloc . So if you overwrite a block you'll get an error the next time you call a heap function (on the same heap) and not only when the system tries the alloc the next block. So, if you see heap corruption this way, you can assume that it occured somewhere between this point and the last time you called a heap function.
Huw.
Huw D M Davies wrote:
On Sun, Sep 01, 2002 at 11:33:55PM +0300, Shachar Shemesh wrote:
Marcus Meissner wrote:
On Sun, Sep 01, 2002 at 12:17:38PM +0300, Shachar Shemesh wrote:
Hi list,
I am currently having some problems with a change I am working on, that result in heap corruption as a result of buffer overrun. I was wondering whether there was any builtin debug options to detect and help fix these things.
If there aren't, I am perfectly willing to sidetrack a little and write them. Please let me know.
If you run with -debugmsg +heap it will check the heaps at every access for corruption.
Ciao, Marcus
Thanks, but this is not what I meant.
Currently, when a buffer overruns, the free pointers right after get corrupted, and when the next piece of block is allocated, the machine will crash. This will tell me approximetly where the buffer overrun, but is, generally speaking, too late. Enabling the debug messages detects the error at the same place (i.e. - too late).
What I am talking about is a compile time (or run time, no reason why not) option that will add "hot zone" areas before and after each heap allocated buffer. These zones should be: A. Large enough so that the overrun doesn't reach the heap managment structures themselves. This means that the program can keep on running despite having the buffer overflow. B. Identifiable, so that we know if a buffer has overwritten it.
No, as Marcus says, enabling +heap debugging forces the entire heap to be checked for consistency on every HeapAlloc/Free/ReAlloc . So if you overwrite a block you'll get an error the next time you call a heap function (on the same heap) and not only when the system tries the alloc the next block. So, if you see heap corruption this way, you can assume that it occured somewhere between this point and the last time you called a heap function.
Huw.
I'm sorry, but my experience is totally different.
First, I would like to say that I have found the offending allocation.
The problem was with not allocating enough memory (enough for the single width string, not the double width string). Classical mistake. The traces proved useless for several reasons: A. The first place the assertation failed is the same place that wine, without the debug option, would crash. This place is far away from the actual allocation OR freeing of the offending buffer. Having the error point at the actual offending buffer will make life much easier for the debugger (IMHO). B. The current mechanism obviously doesn't work well. By the time the crash/assertion happen, both magic and the value before that seem to be valid for the offending pointer block, but both previous and next pointers were values taken from the middle of my buffer (I could recognize the string). This suggests to me that several alloc/free operations happened before the problem was spotted, migrating the overwritten values to other nodes. The mechanism did not catch that. C. The code I am working with is code I previously removed from Wine, and am now trying to restore. In addition to the serious overrun, it also has an "off by one", where a single NULL is written past the end of the buffer. Running the code with debug +heap does not reveal that.
Implementing my suggested system will: A. not hurt the programs operation (despite the overrun), and so the relevant buffer can be found as soon as it is "HeapFree"ed. This means pointing at the actual buffer. B. Even if this is a buffer overrun coupled with a memory leak (HeapFree is NEVER called), we can verify all the hot zones signatures on each heap operation (the same way we do today for the linked list), and give the actual pointer to the offending buffer (which now merely becomes a game of catching who allocated that, almost trivial with +relay)
I'm afraid I won't do it now, as I have found my curlpit, and have bigger itches to scratch. I am suspectful of off by ones creeping up.
Shachar
P.S. Being a security expert as my alter ego, I couldn't help but notice the places where we are vulnerable to integer overflow attacks. For the uninitiated, these are attacks where the attacker minpulates the sizes so that a very big number to be allocated, after being multiplied by the element size, overflows to be a very small number. The attacker can then overrun this small buffer, point the execution path into attacker supplied buffer, and gain control of a program.
In order to solve this we need to supply a function to perform the multiplication for us (HeapAllocElement(Heap, flags, num, element_size) sort of thing), which will check against this condition and then call HeapAlloc. Are there any objections? I know Wine is just Alpha, and noone is supposed to trust it, yadda yadda yadda, but I'm thinking "better early than late".
Sh.
On Mon, Sep 02, 2002 at 12:22:44AM +0300, Shachar Shemesh wrote: [many lines of history deleted]
Implementing my suggested system will:
Maybe there's another solution: Someone with very intricate knowledge of wine and the x86 processor could get valgrind to work - this would provide a very powerful tool.
Ciao Jörg -- Joerg Mayer jmayer@loplof.de I found out that "pro" means "instead of" (as in proconsul). Now I know what proactive means.
Joerg Mayer wrote:
On Mon, Sep 02, 2002 at 12:22:44AM +0300, Shachar Shemesh wrote: [many lines of history deleted]
Implementing my suggested system will:
Maybe there's another solution: Someone with very intricate knowledge of wine and the x86 processor could get valgrind to work - this would provide a very powerful tool.
Ciao Jörg
Just my opinion, of course, but this would require WAY more work than implementing my suggestion. I am also not sure how valgrind does with heaps, where the entire memory does belong to the application, but a given memory location is accessed via a pointer that was supposed to be used on another section.
I know that commercial Win32 tools (such as BoundsChecker, by NuMega) do that using profie compiling - i.e. - they do exactly what I suggested in the previous mail.
As usual, the best approach would probably be doing both.
Shachar
Thanks, but this is not what I meant.
Currently, when a buffer overruns, the free pointers right after get corrupted, and when the next piece of block is allocated, the machine will crash. This will tell me approximetly where the buffer overrun, but is, generally speaking, too late. Enabling the debug messages detects the error at the same place (i.e. - too late).
What I am talking about is a compile time (or run time, no reason why not) option that will add "hot zone" areas before and after each heap allocated buffer. These zones should be: A. Large enough so that the overrun doesn't reach the heap managment structures themselves. This means that the program can keep on running despite having the buffer overflow. B. Identifiable, so that we know if a buffer has overwritten it.
Whenever you free a buffer, you check the hot zones. If they have changed, the buffer you are freeing now has been overwritten. This gives an excellent debugging tool, as it points right at the offending code.
Such a thing is not really implemented yet.
There is a freeware tool called 'valgrind' which might be instrumentable, but I do not think it can cope with wines very own memory management yet.
You could add assert(HeapValidate(heap,0,NULL))); around critical pieces of code, to call the heap checking more often. (replace heap by GetProcessHeap(), or whatever heap you use.)
Additionaly you could careful reread and reaudit your code (even in diff form) ;)
Ciao, Marcus
Marcus Meissner wrote:
Such a thing is not really implemented yet.
Does anyone have any objections to me having a go at implementing it (perhaps at some time in the future)?
There is a freeware tool called 'valgrind' which might be instrumentable, but I do not think it can cope with wines very own memory management yet.
It has a whole bunch of other limitations that do not look promising. Bad replacing stacks, bad own memory managment support (i.e. - we have to teach it how Windows allocates memory), no good support of threads. It seems to me like there is a LOT of porting work to do.
My suggestion, while nowhere near as powerful as that, is simple and with low overhead.
You could add assert(HeapValidate(heap,0,NULL))); around critical pieces of code, to call the heap checking more often. (replace heap by GetProcessHeap(), or whatever heap you use.)
I have found the specific cause of my corruption. I am talking about a "next time" tool, as well as an auditing tool (the most dangarous overruns are those that do not crash the program).
I was also not asking anyone to implement this. Just asking whether anything like that was already implemented, and whether there was any objections to me having a go at implementing it if not.
Shachar
-
Huw D M Davies -
Joerg Mayer -
Marcus Meissner -
Marcus Meissner -
Shachar Shemesh