[Bug 20554] New: Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

Summary: Populous 3 fails to start
           Product: Wine
           Version: 1.1.12
          Platform: PC-x86-64
        OS/Version: Linux
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: winex11.drv
        AssignedTo: [e-mail]
        ReportedBy: [e-mail]
                CC: [e-mail]

Created an attachment (id=24514)
 --> (http://bugs.winehq.org/attachment.cgi?id=24514)
Console output with the bad commit

Populous 3 begins qith wine 1.1.39 but fails with wine 1.1.32. I hav3e made a
regression test and I have found out that

6b8753185f30197047773af6a26eafd34d45367c is first bad commit
commit 6b8753185f30197047773af6a26eafd34d45367c
Author: Roderick Colenbrander 
Date:   Sat Sep 26 14:52:46 2009 +0200

winex11: Add support for 16-bit/32-bit DIB sections.

:040000 040000 f2d2821909d152a1d37ae2932bf6017d7b622344
74221a3bdd0b8be716ed4c7900a58fddc93a2f08 M    dlls

Find attached the console output of populous 3 with the bad commit.

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

Vitaliy Margolen  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |regression

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

--- Comment #1 from Luis Busquets   2009-11-02 11:28:20 ---
Sorry, I ,eamt that it works with wine 1.1.29

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

--- Comment #2 from Roderick Colenbrander   2009-11-02 16:32:43 ---
Is there a demo of this game which shows the same issue? Can you also add a log
using a Wine version with debug symbols (e.g. a Wine version compiled by hand
from Git or so).

Thanks,
Roderick

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

--- Comment #3 from Luis Busquets   2009-11-03 13:33:47 ---
Attached the debug with trace on for all

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

--- Comment #4 from Luis Busquets   2009-11-03 19:23:06 ---
Created an attachment (id=24539)
 --> (http://bugs.winehq.org/attachment.cgi?id=24539)
debug with trace on for x11drv

I tried to upload the file with trace on for all but this files is 55 MB and it
failed

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

--- Comment #5 from Luis Busquets   2009-11-03 19:36:46 ---
Created an attachment (id=24540)
 --> (http://bugs.winehq.org/attachment.cgi?id=24540)
Debug with trace on for bitmap

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

--- Comment #6 from Luis Busquets   2009-11-03 19:47:51 ---
Created an attachment (id=24541)
 --> (http://bugs.winehq.org/attachment.cgi?id=24541)
Cut of debug log last lines before the output "wine: Unhandled..."

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

--- Comment #7 from Luis Busquets   2009-11-03 20:33:21 ---
Is that the information required? what is the correct way to compile it with
debug symbols? I thought this was done  by default in the git, do I have to run
any special command before compiling the git tree?

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

--- Comment #8 from Roderick Colenbrander   2009-11-04 13:16:36 ---
I can't see that much in the log. From a quick glance it looks similar to the
issue seen in Heroes of Might and Magic III but also there the exact issue is
very hard to find. If the issue is the same (and I'm still not sure what the
issue is) it is likely a bug which my change uncovered.

I tried the demo available at
http://www.brothersoft.com/games/populous-the-beginning-download.html and that
one works for me. Can you check if the demo shows the same issue for you?

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

--- Comment #9 from Luis Busquets   2009-11-04 20:49:22 ---
I have checked the demo and for me it also works with wine 1.1.32. So the demo
does not include the problem.

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

--- Comment #10 from Luis Busquets   2009-11-04 21:25:08 ---
Created an attachment (id=24558)
 --> (http://bugs.winehq.org/attachment.cgi?id=24558)
Modified wine to provide more info

I have included fixmes in the functions that the patch tackles to see when the
program calls and when they return.

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

--- Comment #11 from Roderick Colenbrander   2009-11-05 04:16:15 ---
Let me quickly explain the problem. Before this patch we only created X Pixmaps
for DIBSections in 1-bit and the depth of the Xserver, so usually 24-bit. When
a program requested 16-bit we would still allocate a 24-bit Pixmap and under
the hood perform depth conversion.

The reason for not creating Pixmaps in other depths was that X doesn't allowing
blitting e.g. 16-bit data to 24-bit. XRender does allow this and I did a lot of
work to prepare the Wine code for this. The only places where XRender is really
used is during blitting and font rendering.

The final patch for the XRender work (the one which broke your game) added
X11DRV_XRender_SetPhysBitmapDepth which is used for setting the depth of a
DIBSections to any depth support by XRender, so typically 1-bit, 16-bit, 24-bit
and 32-bit.

I fear that some other code e.g. ddraw/wined3d or some other part of
winex11.drv contains some bugs which are uncovered by this. I have no idea
where to search further yet.

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

--- Comment #12 from Roderick Colenbrander   2009-11-05 09:05:13 ---
I did some testing myself. WineD3D calls CreateDIBSection from
IWineD3DBaseSurfaceImpl_CreateDIBSection. In that function it asks
CreateDIBSection to allocate 640x481x16 (481 because it needs an extra line).
On my system the allocate memory is at 0x3110000. (+d3d_surface shows that)

The game crashes in a similar way as on your system:
First chance exception: page fault on write access to 0x03124000 in 32-bit code
(0x0057d7c9).
Register dump:
 CS:0023 SS:002b DS:002b ES:002b FS:0063 GS:006b
 EIP:0057d7c9 ESP:0033f0f0 EBP:03124000 EFLAGS:00010216(  R- --  I   -A-P- )
 EAX:00200840 EBX:00000800 ECX:00000000 EDX:000009fe
 ESI:2e023a72 EDI:0033f4f0
Stack dump:
0x0033f0f0:  0000000a 0000000b 0000000b 0000000b
0x0033f100:  0000000a 0000000a 0000000a 0000000b
0x0033f110:  0000000c 0000000a 0000000a 0000000a
0x0033f120:  00000009 00000008 00000008 00000009
0x0033f130:  0000000a 0000000b 0000000b 0000000b
0x0033f140:  0000000a 0000000a 0000000a 0000000b
Backtrace:
=>0 0x0057d7c9 in d3dpoptb (+0x17d7c9) (0x03124000)

The memory 0x03124000 is part of the range allocate for the DIBSection namely
0x03110000 - 0x035c2800 (0x03110000+640*481*16). So apparently something goes
wrong there.

The problem that it is VERY hard to debug. DIBSections can be modified by
directly poking with bits from a program and they can also be modified by GDI
calls (in that case a DIBSection is drawn to using X functions). Segmentation
faults are used to keep both copies in sync.

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

--- Comment #13 from Alexandre Julliard   2009-11-05 09:21:03 ---
(In reply to comment #12)
> I did some testing myself. WineD3D calls CreateDIBSection from
> IWineD3DBaseSurfaceImpl_CreateDIBSection. In that function it asks
> CreateDIBSection to allocate 640x481x16 (481 because it needs an extra line).
> On my system the allocate memory is at 0x3110000. (+d3d_surface shows that)
>
> The game crashes in a similar way as on your system:
> First chance exception: page fault on write access to 0x03124000 in 32-bit code

That's a first chance exception so it's most likely not the problem. It will be
handled by the DIB section handler.

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

--- Comment #14 from Roderick Colenbrander   2009-11-05 10:50:47 ---
I have come a little bit closer. Disabling the BitBlt in swapchain_gdi.c
x11_copy_to_screen prevents the crash. I checked X11DRV_StretchBlt and each
time it is called it also completes. Perhaps somehow it leaves something in a
wrong state behind or so.

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

--- Comment #15 from Luis Busquets   2009-11-05 21:57:33 ---
Please post the patch that disables it so that I can test and verify that is
the same problem.

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

--- Comment #16 from Luis Busquets   2009-11-06 05:43:23 ---
I do not know if this will help but I have noticed that you say 481. Well
Actually in the trace that I have there are always pairs:

trace:bitmap:CreateDIBSection format (640,-481), planes 1, bpp 16, size 615680,
RGB
trace:bitmap:CreateBitmapIndirect 640x481, 1 colors returning 0x530

I assume that the first function calls at some point the second with -(-481).

Why does the first -481 appear? Actually binary speaking 480 = not(-481) So at
some point somebody has done binaryli the complement to 1 of 480.

Anyway it is strange to try to create a DIB with (640, -481) a negative value,
is it?

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

--- Comment #17 from Luis Busquets   2009-11-06 08:04:59 ---
I found this in the MSDN
"biHeight

The height of the bitmap, in pixels. If biHeight is positive, the bitmap is
a bottom-up DIB and its origin is the lower-left corner. If biHeight is
negative, the bitmap is a top-down DIB and its origin is the upper-left corner.

If biHeight is negative, indicating a top-down DIB, biCompression must be
either BI_RGB or BI_BITFIELDS. Top-down DIBs cannot be compressed.

If biCompression is BI_JPEG or BI_PNG, the biHeight member specifies the
height of the decompressed JPEG or PNG image file, respectively."

Is the function CreateDIBImage acting correctly when the bitmap has a negative
height? Is that implemented?

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

--- Comment #18 from Roderick Colenbrander   2009-11-07 09:29:59 ---
The DIB functions should be working correctly. Nearly all Windows programs use
DIBs. The main thing which now changed is that now most DIBSections are backed
by a Pixmap at the same depth. I guess somewhere some of the DIB code is just
broken but I can't see what. Perhaps I can get AJ to look at it here on my
laptop at WineConf.

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

Roderick Colenbrander  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|winex11.drv                 |directx-ddraw

--- Comment #19 from Roderick Colenbrander   2009-11-08 08:47:44 ---
At WineConf I looked at this bug with Henri and in the end Alexandre joined us
as well. As I mentioned before we use segmentation faults for syncing the
memory of DIBSection with the X11 copy. Normally this works well. After
disassembling the app it appeared that for some reason Populous is messing with
the stack pointer and due to this we lose the stack and crash. For some reason
we got away with this before and now it broke. Alexandre said that the only
real solution to the issue would be a DIB engine in which we don't have to use
this evil memory tricks.

We could copy the DIB data over to a new buffer (in ddraw) and sync that buffer
later on but memcpy operations are very expensive and it would really kill
performance of other games. The issue we are seeing in Heroes 3 might be
related but I'm not sure.

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

--- Comment #20 from Luis Busquets   2009-11-09 14:52:57 ---
Created an attachment (id=24626)
 --> (http://bugs.winehq.org/attachment.cgi?id=24626)
bitmap debug adding to the source code additional TRACE

This is the debug when adding to the source additional trace so that it shoes
info for X11DRV_CreateDIBSection on which is the assigned
physBitmap->pixmap_depth , physBitmap->trueColor and
physBitmap->pixmap_color_shifts

This output iswith source code line

if (!X11DRV_XRender_SetPhysBitmapDepth( physBitmap, &dib ))

is commented and then the programme works without problem.

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

--- Comment #21 from Luis Busquets   2009-11-09 14:54:25 ---
Created an attachment (id=24627)
 --> (http://bugs.winehq.org/attachment.cgi?id=24627)
bitmap debug with additional trace

This is the debug when adding to the source additional trace so that it shoes
info for X11DRV_CreateDIBSection on which is the assigned
physBitmap->pixmap_depth , physBitmap->trueColor and
physBitmap->pixmap_color_shifts

This output is with source code line

if (!X11DRV_XRender_SetPhysBitmapDepth( physBitmap, &dib ))

not commented and then the programme crashes.

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

--- Comment #22 from Luis Busquets   2009-11-10 22:53:12 ---
Created an attachment (id=24665)
 --> (http://bugs.winehq.org/attachment.cgi?id=24665)
bitmap with additional trace without 16 bit support

Instead of de-activating completely X11DRV_XRender_SetPhysBitmapDepth as in the
previous bitmap ç(which then the Populous does not crash), I have added to that
function:

if (fmt->pict_format->depth==16) return FALSE;

To X11DRV_XRender_SetPhysBitmapDept

So that when it finds 16-bit pixmap, it falls back to the ancient source coude.
This way, 32--bit DIBS are still active and Populous works perfectly. Does that
mean that the sync problem only appears in 16-bit dibs?

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

--- Comment #23 from Luis Busquets   2009-11-10 22:55:18 ---
Would it be possible to include a patch with

if (fmt->pict_format->depth==16) return FALSE;

so that this bug is resolved?

The previous patch still adds 32 functionality.

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

--- Comment #24 from Luis Busquets   2009-11-10 23:00:55 ---
Created an attachment (id=24666)
 --> (http://bugs.winehq.org/attachment.cgi?id=24666)
De-activate 16 bit pixmaps and add additional TRACEs

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

--- Comment #25 from Roderick Colenbrander   2009-11-11 03:00:08 ---
Adding a patch which would disable XRender for 16-bit is not an option. For
most other apps the 16-bit patch gives a nice speedup. I'm not sure but I guess
the reason it worked before is that we used no (?) access traps because the
depths were different but I'm not sure (that's the only thing I can think of
why it would have worked).

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

--- Comment #26 from Luis Busquets   2009-11-11 09:26:56 ---
An d what about a patch that returns FALSE if the color format is R5G6B5? 
Maybe that is the color format which has got the bug in WINE?

Would that be acceptable?

Would that information be relevant to find the wine bug?

Which 16 bits formats have used applications used with this dib patch without
crashing?

Is R5G6B5 between them?

BTW, I assume that this is the format that is using populous

physBitmap->pixmap_color_shifts ((11, 5,31), (0, 5, 31), (5, 6, 63), (11,
5,31), (0, 5, 31), (5, 6, 63))

if not, please correct.

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

--- Comment #27 from Roderick Colenbrander   2009-11-11 09:33:33 ---
I have used that format properly in for instance the demo of C&C Tiberian Sun.
Just download and it should work fine. Please check it if you can.

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

--- Comment #28 from Luis Busquets   2009-11-11 14:53:56 ---
I have tried Tiberian Sunb and for me certainly works. I wonder what the bug
is!!! This is going to be extremely tough.

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

--- Comment #29 from Roderick Colenbrander   2009-12-11 13:05:16 ---
I have attached a patch to bug 20380 which should fix the populous bug.

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

--- Comment #30 from Luis Busquets   2010-01-15 16:43:04 ---
I have tried the patch that you propose for populous 3 in bug 20380 and it
solves the issue. Are you planning to commit it?

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

quasi-anonymous user from bugmenot.com  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
     Ever Confirmed|0                           |1

--- Comment #31 from quasi-anonymous user from bugmenot.com   2010-01-23 12:27:16 ---
*** This bug has been confirmed by popular vote. ***

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

--- Comment #32 from Luis Busquets   2010-03-10 00:25:25 ---
Roderickm are you planning to commit that patch?

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

--- Comment #33 from Roderick Colenbrander   2010-03-10 01:44:28 ---
No, I won't add it to Wine. It is worth a test but check out bug 20380. Sure
the populous code is really broken but you might get away with it you use the
Linux kernel patch which is mentioned in that bug. There is a Linux bug which
is related to similar problems.

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

GyB  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #34 from GyB  2010-11-22 13:13:54 CST ---
Hello,

I accidentally stumbled on a small, freeware game (Klabutong). The game
received Platinum rating with 1.0.1 but seems to be broken with the current git
(wine-1.3.7-172-g8925024): it crashes on startup.
Regression testing resulted the same commit as reported in this bug report:

commit 6b8753185f30197047773af6a26eafd34d45367c
winex11: Add support for 16-bit/32-bit DIB sections.

I just thought this might be the same issue because Rodericks's patch for
Populous 3 (in bug #20380#c45) fixes the crash in this game, too.

Link to the download page of the game:
http://www.freelunchdesign.com/games.php?id=8
(only 3 MB)

Fedora 14
Nvidia 7600 / driver 260.19.21

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

Austin English  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |download
                URL|                            |http://www.freelunchdesign.
                   |                            |com/games.php?id=8

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

joaopa  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #35 from joaopa  2011-03-21 13:18:13 CDT ---
Is this bug fixed with wine 1-3-16?

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

Henri Verbeet  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
    Regression SHA1|                            |6b8753185f30197047773af6a26
                   |                            |eafd34d45367c

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

--- Comment #36 from GyB  2011-10-15 00:58:11 CDT ---
I don't have Populous 3 so can't tell you whether this bug is fixed or not
regarding to P3, but Klabutong (see comment #34) is starting properly now, as
of wine-1.3.30-145-g7a4349b.
Klabutong began to work on my system after OpenGL became the default renderer
(1.3.28). However, gdi still produced the same crash.
Since this commit, the game starts correctly, whether gdi or opengl is in use:
http://source.winehq.org/git/wine.git/commit/0f9dfb932e51efa844283d28d7aa9be...

Please try Populous 3 with Wine-1.3.29 (or newer) and report back what you
find.

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

Austin English  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |FIXED

--- Comment #37 from Austin English  2011-10-17 18:30:35 CDT ---
Works for me in wine-1.3.30-205-g472a8f7. Presumably something in DIB engine
fixed it.

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

Alexandre Julliard  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |CLOSED

--- Comment #38 from Alexandre Julliard  2011-10-21 13:49:43 CDT ---
Closing bugs fixed in 1.3.31.

[Bug 20554] Populous 3 fails to start

http://bugs.winehq.org/show_bug.cgi?id=20554

Austin English  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|directx-ddraw               |directx-d3d