Help debugging a problem!

[Ann & Jason Edmeades]

Hiya,

I've got a weird issue and was wondering if anyone could advise on how to
resolve. (Comes from a 16bit windows app, but is a more general debugging
issue).

The problem is ...
1) If I run the application, it just hangs - no overly helpful information
at all.
2) If I add WINEDEBUG +relay trace, it runs ok :-)  (Workaround...!)
3) If I add anything else on WINEDEBUG other than relay, it fails
4) Running under winedbg didn't help at all - I could code to stop it before
the hang but it wouldn't step through to the failure
5) Attaching to the hung process shows code in 16 bit user app, and winedbg
didn't overly help there either

So - I've (painstakingly) got to the point of failure with the add of debug
tracepoints and comparing the relay with the debug statements.

Now the odd bit!

If you look at dlls\user\wnd16.c, routine RegisterClass16 the final line is:
    return RegisterClassEx16( &wcex );

If I change this to
    fred = RegisterClassEx16( &wcex );
    TRACE("Here... %d\n", fred);
    return fred;

it all works.....!

How can I debug this further? I was thinking about trying to dump the
registers before and after the trace statement, but I really can't think of
what could be causing the problem!

Does windows guarantee any of the registers across a win16 call which we
don't honour? What about i386 flags?

Any suggestions please?

Jason

[James Hawkins]

On 9/27/05, Ann & Jason Edmeades  wrote:
> If I change this to
>     fred = RegisterClassEx16( &wcex );
>     TRACE("Here... %d\n", fred);
>     return fred;
>
> it all works.....!

This sounds like the stack is getting trashed.  That is usually the
case when randomly added functions make it not crash.  You want to
look for places where local variables (on the stack of course) are
being overwritten.  A good example of this is copying a larger string
into a smaller local char array.  When looking for a stack crasher, I
comment out huge sections of code until it doesn't crash anymore, and
then piece by piece uncomment the next function.  For example,

void foo()
{
#if 0
    a();
    b();
    c();
#endif
}

doesn't crash, but

void foo()
{
    a();
#if 0
    b();
    c();
#endif
}

does crash, then your culprit is in a().  Then you recurse into a()
and repeat the process all over again till you find the stack crasher.

--
James Hawkins

[James Hawkins]

On 9/27/05, James Hawkins  wrote:
> On 9/27/05, Ann & Jason Edmeades  wrote:
> > If I change this to
> >     fred = RegisterClassEx16( &wcex );
> >     TRACE("Here... %d\n", fred);
> >     return fred;
> >
> > it all works.....!
>
> This sounds like the stack is getting trashed.

I just read through your post again and saw that the app was hanging,
and by hanging I'm assuming it's stuck in the code somewhere, whereas
I thought the app was crashing.  It might not be the case then that
the stack is getting trashed, though you can still use the debugging
technique mentioned.

--
James Hawkins

[Ivan Leo Puoti]

James Hawkins wrote:
> On 9/27/05, Ann & Jason Edmeades  wrote:
> > If I change this to
> >    fred = RegisterClassEx16( &wcex );
> >    TRACE("Here... %d\n", fred);
> >    return fred;
> >
> > it all works.....!
>
> This sounds like the stack is getting trashed.

I agree this looks very much like a stack corruption issue.

Ivan.