February 2009 - Wine-Devel - list.winehq.org

[patch but not quite ready] NamedPipes messagemode

by Luke Kenneth Casson Leighton

http://bugs.winehq.org/show_bug.cgi?id=17195 with only one significant bug - some memory corruption - the semantics are now correct in the namedpipe messagemode patch. i drew a diagram outlining the data structures / design http://bugs.winehq.org/attachment.cgi?id=19449 because it's so horrendous that even i want to throw up. assistance in finding the memory corruption would be really helpful. an assert is thrown after more than 4 short reads are performed on the same namedpipe, with this straaaange error: ead: file.c:631: server_get_named_pipe_fd: Assertion `wine_server_ptr_handle(fd_handle) == handle' failed. ead: server.c:628: server_get_unix_fd: Assertion `wine_server_ptr_handle(fd_handle) == handle' failed. wine: Assertion failed at address 0xf7ca8d7f (thread 000f), starting debugger... err:ntdll:RtlpWaitForCriticalSection section 0x7bc99ee0 "server.c: fd_cache_section" wait timed out in thread 000f, blocked by 0014, retrying (60 sec) i'll try to write a simplified test because that's the "threadread" test on which this occurs. i can't quite believe that all 5 tests can be passed [by reducing the number of short reads to less than 4]. it's been a bit of a "drunken walk" development style, i'm afraid :) notes: * in order to preserve message-mode semantics correctly, each message gets ITS OWN socketpair. the socketpair is used EXCLUSIVELY for that one message and is NOT re-used by the other end. so if it's client writing to server, one message, that's one socketpair. if it's server writing to client, one message, that's another socketpair. if client writes 20 messages to server, that's 20 new socketpairs, if at same time server writes 25 messages to client, and nobody's done any reading yet, that's a total of 45 outstanding socketpairs - 90 filedescriptors. this is not a hoax. it's the only way i could think of that preserves messagemode semantics correctly. * for each end - client and server - to be able to "block", waiting at any time, there MUST be at LEAST one socketpair in each of the queues. so, initially, that's what's done: one pair of socketpairs are created, one for the client, one for the server. on the FIRST write (to server, or to client), this initial "empty" socketpair must be written to; subsequent writes (to server or to client, each) must have a NEW socketpair created. * on a read, a socketpair must "hang about" - MUST not be shut down - until there is a new one created (which happens only on a new write). in this way, multiple threads will always have a filedescriptor to block on. * when a read has read all available data, the socketpair is marked as "finished" but is NOT shut down. * on each write, if a new socketpair is created, then and ONLY then can "old" marked socketpairs be shut down. * shutdown of read-sockets, on which [potentially many] reader[s] are waiting will result in an EPIPE. this is an ACCEPTABLE error IFF no data has been read yet, i.e. at the start of a new message. what happens, basically, is that the "old" socketpair - even though it had been used already by the previous reader - is what the thread "blocks" on (because you have to block on _something_). and on the _very_ first message, of course, you block on the specially-prepared initial socketpair, the one that had to be created and marked as "as-yet unwritten to". * you can see at line 872 of ntdlls/file.c (after patch is applied) that oh whoopsie, EPIPE occurred, but we've not read any data at all, just grab another filedescriptor from wineserver and try again. in this way, wineserver hands out the next available message from the filedescriptor queue. if yet _another_ EPIPE occurs, then all that means is that another thread "got there first", wow big deal, around you go again. * indicating that data has been read is performed using a special wineserver message "NAMED_PIPE_SUB_AVAILDATA" using server_set_named_pipe_info(). this horrible hack passes information to wineserver telling it to subtract the amount of data read in userspace from the filedescriptor data structure (fdqueue) representing that socketpair. * indicating that data _will_ be available is performed again using server_set_named_pipe_info(), this time using a message "NAMED_PIPE_SET_AVAILDATA". the set is performed BEFORE the actual write, otherwise a reader will "unblock" and potentially try to set the available data to negative!!! this would be baaad. * a critical section is performed around all uses of server_set_named_pipe_info() and server_get_named_pipe_fd(). this is NOT the same critical section as server_get_handle_fd(). i don't know if this makes any difference. things that cannot be done: * you cannot do the read or the write inside wineserver (server_named_pipe_read() will be deleted, later). the reason is complex to describe, but boils down to "blocking in userspace on the socket in order to wait for a message" and "non-blocking read on the same socket in order to get the data" are unfortunately mutually incompatible. one screws up the other if you set or clear O_NONBLOCK. * you cannot send the length as a 4-byte "header" down the socket, to be read as a length and then that much data read out. if you have multiple readers blocking on the same filedescriptor, how do you make sure that only one of them reads the length?? they're all blocking on the same socket! send the length multiple times?? how do you know how many readers there are?? short answer: you can't. so - scratch that idea. [ok - maybe you could do a critical section, but you'd have to do one per file, and i'm not ready or willing to add an array of per-file-handle critical sections to ntdll without further discussion] issues: * multiple readers of the same socket, when reading data amounts that are larger than the SO_RCVBUF and/or SO_SNDBUF, could potentially end up corrupting the data because of overlaps in the reads. given that this is _already_ a problem, in the existing Wine code, i am not going to be dealing with this issue, in this iteration. particularly as solutions involve having to do per-file critical sections and YES that's a generic problem even on standard files not just pipes not just messagemode pipes ALL files. * running out of socketpairs; protecting the message queue length. i believe there's a default message queue length in NT, but even so, the number of file descriptors this [necessary] design uses up is _horrendous_. one pair per message! but - tough luck. the options are pretty limited and constrained by the requirements. so: if anyone says to me, "no this cannot work, you cannot _possibly_ have got correct messagemode semantics purely in userspace and without sending length fields, you _must_ do the reads and writes inside wineserver", then... well... that will be a good way to get me to unsubscribe from wine-devel. if anyone can think of any other ideas, or implementations, or optimisations, then... well... unless they're _really_ good ideas, they'll have to wait :) ideas such as using a separate socketpair for communicating "length" messages are welcome, if accompanied by well-laid-out explanations, because my brain is turning to jelly with this stuff and i could do with some light reading instead of working it out myself :) l.

Feb. 14, 2009

Re: 1.1.15 failing to build -- what changed?

by James McKenzie

Ben Klein wrote: > Stupid Gmail sent my last mail twice, it seems. > > 2009/2/15 Roderick Colenbrander <thunderbird2k(a)gmx.net>: > >>> 1.1.15 fails to build using an identical layout to 1.1.14, which means >>> something has changed with either the build dependencies or the way >>> configure is working. >>> >>> Here's the build failure I'm getting on amd64, but it fails on all >>> arches: >>> http://launchpadlibrarian.net/22606029/buildlog_ubuntu-intrepid-amd64.wine_… >>> >>> I'm going to sleep now, but hopefully I'll figure this out Valentine's >>> day. In the meanwhile, 1.1.15 won't be available in Ubuntu packages >>> just yet. >>> >>> Thanks, >>> Scott Ritchie >>> >>> >> A packager of a debian based distro reported the same issue. They are doing: >> CFLAGS="$(CFLAGS)" LDFLAGS="$(LDFLAGS)" ./configure --host=$(DEB_HOST_GNU_TYPE) --build=$(DEB_BUILD_GNU_TYPE) --prefix=/usr --mandir=\$${prefix}/share/man --infodir=\$${prefix}/share/info $(CONFFLAGS) >> >> Where the two DEB_* variables are: >> DEB_HOST_GNU_TYPE ?= $(shell dpkg-architecture -qDEB_HOST_GNU_TYPE) >> DEB_BUILD_GNU_TYPE ?= $(shell dpkg-architecture -qDEB_BUILD_GNU_TYPE) >> >> When building 32-bit packages DEB_HOST_GNU_TYPE is set to i486-linux-gnu and this results in 'i486-linux-gnu-as not found'. >> > > This is interesting. I don't do this in my build system; I have > chroots of each distro I build and a script that automates most of the > work. I just run ./configure --verbose, pretty much, but I've > installed a bunch of optional libs (like gphoto and such). > > >> Personally I consider it a misconfigured build system. It can be fixed by installing the proper binutils package for this architecture which I'm sure debian has. The same should be done for your ubuntu packages. You can't expect apps to build if you only have one half of the build apps (the compiler). >> > > That's what I was thinking too. Broken and/or misconfigured build environment. > > > > Please be aware that my MacOSX build based on Mike Kronenberg's build environment completed without any errors. James McKenzie

Feb. 14, 2009

Re: mshtml: allow REG_EXPAND_SZ values for GeckoCabDir

by Juan Lang

> No, I used RegGetValue which expands them for me. Ah, right, that explains the other change as well. Thanks :) --Juan

Feb. 14, 2009

Re: Proposal: put Winetricks into Wine menu

by Rosanne DiMesio

> > Now, at some point we should come up with a better menu item name > than Winetricks, but I have no idea how to sum it up properly. > winetricks has a fair bit of name recognition, and requires no > localization, so let's run with that name until we figure out > what it should really be called, and what the GUI really > should look like in the end. I think it would be better to stick with winetricks, precisely because of the name recognition. Think of all the forum posts, AppDB entries, and wiki entries that mention winetricks. Change the name and we're going to have to endlessly explain that winetricks is now called something else. -- Rosanne DiMesio <dimesio(a)earthlink.net>

Feb. 14, 2009

Re: Proposal: put Winetricks into Wine menu

by Dan Kegel

Great! If Scott does the same for Ubuntu, it'll simplify my CeBit presentation. Currently I have to dance around explaining how to get winetricks... if it's right there in the menus, the presentation will flow better. On Sat, Feb 14, 2009 at 7:23 AM, marco <marco(a)mandrivaclub.nl> wrote: > A menu item is added for mandriva > and dependencies are added. > > and will be active for the next version 1.1.16 > > Marco > > > Dan Kegel schreef: >> >> So, for people packaging wine, here's a proposed plan: >> >> 0. See http://wiki.winehq.org/winetricks if you're not familiar with >> winetricks >> >> 1. Download http://www.kegel.com/wine/winetricks >> (or get it via svn, see http://code.google.com/p/winezeug ) >> It's a monolithic shell script, so you don't need to compile it or >> anything. >> >> 2. Move it to /usr/bin and do chmod +x /usr/bin/winetricks >> >> 3. Add a dependecy on cabextract, sha1sum, and wget or curl. >> >> 4. In your package's changelog, note which version of Winetricks >> you pulled down. You'll want to pull down a fresh one every time >> you package Wine, as it contains download URLs that occasionally >> go stale. (In particular, I need to update the gecko URL for wine-1.1.5, >> I'll do that today.) >> >> The Suse packages already do all the above steps, I think; >> you might ask Marcus Meissner if you want to confirm the details. >> >> Here's the part that I don't think anybody does yet: >> >> 5. Add a menu item under Applications / Wine that runs >> /usr/bin/winetricks. >> In Ubuntu, the existing menu items are >> Programs >> Browse C: Drive >> Configure Wine >> Uninstall Wine Software >> Assuming we name the menu item Winetricks, the menu would then look like >> Programs >> Browse C: Drive >> Configure Wine >> Uninstall Wine Software >> Winetricks >> >> Now, at some point we should come up with a better menu item name >> than Winetricks, but I have no idea how to sum it up properly. >> winetricks has a fair bit of name recognition, and requires no >> localization, so let's run with that name until we figure out >> what it should really be called, and what the GUI really >> should look like in the end. >> >> (We also need a menu item there for "Help Using Wine", but >> nobody's written that yet. It can't just be links to the FAQ, >> since it has to work while offline.) >> >> Sound good? >> - Dan >> >> > >

Feb. 14, 2009

Re: Proposal: put Winetricks into Wine menu

by marco

A menu item is added for mandriva and dependencies are added. and will be active for the next version 1.1.16 Marco Dan Kegel schreef: > So, for people packaging wine, here's a proposed plan: > > 0. See http://wiki.winehq.org/winetricks if you're not familiar with winetricks > > 1. Download http://www.kegel.com/wine/winetricks > (or get it via svn, see http://code.google.com/p/winezeug ) > It's a monolithic shell script, so you don't need to compile it or anything. > > 2. Move it to /usr/bin and do chmod +x /usr/bin/winetricks > > 3. Add a dependecy on cabextract, sha1sum, and wget or curl. > > 4. In your package's changelog, note which version of Winetricks > you pulled down. You'll want to pull down a fresh one every time > you package Wine, as it contains download URLs that occasionally > go stale. (In particular, I need to update the gecko URL for wine-1.1.5, > I'll do that today.) > > The Suse packages already do all the above steps, I think; > you might ask Marcus Meissner if you want to confirm the details. > > Here's the part that I don't think anybody does yet: > > 5. Add a menu item under Applications / Wine that runs /usr/bin/winetricks. > In Ubuntu, the existing menu items are > Programs > Browse C: Drive > Configure Wine > Uninstall Wine Software > Assuming we name the menu item Winetricks, the menu would then look like > Programs > Browse C: Drive > Configure Wine > Uninstall Wine Software > Winetricks > > Now, at some point we should come up with a better menu item name > than Winetricks, but I have no idea how to sum it up properly. > winetricks has a fair bit of name recognition, and requires no > localization, so let's run with that name until we figure out > what it should really be called, and what the GUI really > should look like in the end. > > (We also need a menu item there for "Help Using Wine", but > nobody's written that yet. It can't just be links to the FAQ, > since it has to work while offline.) > > Sound good? > - Dan > >

Feb. 14, 2009

Re: Proposal: put Winetricks into Wine menu

by Dan Kegel

So, for people packaging wine, here's a proposed plan: 0. See http://wiki.winehq.org/winetricks if you're not familiar with winetricks 1. Download http://www.kegel.com/wine/winetricks (or get it via svn, see http://code.google.com/p/winezeug ) It's a monolithic shell script, so you don't need to compile it or anything. 2. Move it to /usr/bin and do chmod +x /usr/bin/winetricks 3. Add a dependecy on cabextract, sha1sum, and wget or curl. 4. In your package's changelog, note which version of Winetricks you pulled down. You'll want to pull down a fresh one every time you package Wine, as it contains download URLs that occasionally go stale. (In particular, I need to update the gecko URL for wine-1.1.5, I'll do that today.) The Suse packages already do all the above steps, I think; you might ask Marcus Meissner if you want to confirm the details. Here's the part that I don't think anybody does yet: 5. Add a menu item under Applications / Wine that runs /usr/bin/winetricks. In Ubuntu, the existing menu items are Programs Browse C: Drive Configure Wine Uninstall Wine Software Assuming we name the menu item Winetricks, the menu would then look like Programs Browse C: Drive Configure Wine Uninstall Wine Software Winetricks Now, at some point we should come up with a better menu item name than Winetricks, but I have no idea how to sum it up properly. winetricks has a fair bit of name recognition, and requires no localization, so let's run with that name until we figure out what it should really be called, and what the GUI really should look like in the end. (We also need a menu item there for "Help Using Wine", but nobody's written that yet. It can't just be links to the FAQ, since it has to work while offline.) Sound good? - Dan

Feb. 14, 2009

Re: 1.1.15 failing to build -- what changed?

by Ben Klein

Stupid Gmail sent my last mail twice, it seems. 2009/2/15 Roderick Colenbrander <thunderbird2k(a)gmx.net>: >> 1.1.15 fails to build using an identical layout to 1.1.14, which means >> something has changed with either the build dependencies or the way >> configure is working. >> >> Here's the build failure I'm getting on amd64, but it fails on all >> arches: >> http://launchpadlibrarian.net/22606029/buildlog_ubuntu-intrepid-amd64.wine_… >> >> I'm going to sleep now, but hopefully I'll figure this out Valentine's >> day. In the meanwhile, 1.1.15 won't be available in Ubuntu packages >> just yet. >> >> Thanks, >> Scott Ritchie >> > > A packager of a debian based distro reported the same issue. They are doing: > CFLAGS="$(CFLAGS)" LDFLAGS="$(LDFLAGS)" ./configure --host=$(DEB_HOST_GNU_TYPE) --build=$(DEB_BUILD_GNU_TYPE) --prefix=/usr --mandir=\$${prefix}/share/man --infodir=\$${prefix}/share/info $(CONFFLAGS) > > Where the two DEB_* variables are: > DEB_HOST_GNU_TYPE ?= $(shell dpkg-architecture -qDEB_HOST_GNU_TYPE) > DEB_BUILD_GNU_TYPE ?= $(shell dpkg-architecture -qDEB_BUILD_GNU_TYPE) > > When building 32-bit packages DEB_HOST_GNU_TYPE is set to i486-linux-gnu and this results in 'i486-linux-gnu-as not found'. This is interesting. I don't do this in my build system; I have chroots of each distro I build and a script that automates most of the work. I just run ./configure --verbose, pretty much, but I've installed a bunch of optional libs (like gphoto and such). > Personally I consider it a misconfigured build system. It can be fixed by installing the proper binutils package for this architecture which I'm sure debian has. The same should be done for your ubuntu packages. You can't expect apps to build if you only have one half of the build apps (the compiler). That's what I was thinking too. Broken and/or misconfigured build environment.

Feb. 14, 2009

Re: 1.1.15 failing to build -- what changed?

by Roderick Colenbrander

> 1.1.15 fails to build using an identical layout to 1.1.14, which means > something has changed with either the build dependencies or the way > configure is working. > > Here's the build failure I'm getting on amd64, but it fails on all > arches: > http://launchpadlibrarian.net/22606029/buildlog_ubuntu-intrepid-amd64.wine_… > > I'm going to sleep now, but hopefully I'll figure this out Valentine's > day. In the meanwhile, 1.1.15 won't be available in Ubuntu packages > just yet. > > Thanks, > Scott Ritchie > A packager of a debian based distro reported the same issue. They are doing: CFLAGS="$(CFLAGS)" LDFLAGS="$(LDFLAGS)" ./configure --host=$(DEB_HOST_GNU_TYPE) --build=$(DEB_BUILD_GNU_TYPE) --prefix=/usr --mandir=\$${prefix}/share/man --infodir=\$${prefix}/share/info $(CONFFLAGS) Where the two DEB_* variables are: DEB_HOST_GNU_TYPE ?= $(shell dpkg-architecture -qDEB_HOST_GNU_TYPE) DEB_BUILD_GNU_TYPE ?= $(shell dpkg-architecture -qDEB_BUILD_GNU_TYPE) When building 32-bit packages DEB_HOST_GNU_TYPE is set to i486-linux-gnu and this results in 'i486-linux-gnu-as not found'. Personally I consider it a misconfigured build system. It can be fixed by installing the proper binutils package for this architecture which I'm sure debian has. The same should be done for your ubuntu packages. You can't expect apps to build if you only have one half of the build apps (the compiler). Roderick -- Jetzt 1 Monat kostenlos! GMX FreeDSL - Telefonanschluss + DSL für nur 17,95 Euro/mtl.!* http://dsl.gmx.de/?ac=OM.AD.PD003K11308T4569a

Feb. 14, 2009

Re: 1.1.15 failing to build -- what changed?

by Ben Klein

2009/2/14 Scott Ritchie <scott(a)open-vote.org>: > 1.1.15 fails to build using an identical layout to 1.1.14, which means > something has changed with either the build dependencies or the way > configure is working. Worked for me on my Debian build environments. No changes required ... > Here's the build failure I'm getting on amd64, but it fails on all > arches: > http://launchpadlibrarian.net/22606029/buildlog_ubuntu-intrepid-amd64.wine_… This is the snippet that includes the failure: make[3]: Entering directory `/build/buildd/wine-1.1.15~winehq0~ubuntu~8.10/dlls/acledit' x86_64-linux-gnu-gcc -m32 -c -I. -I. -I../../include -I../../include -D__WINESRC__ -D_REENTRANT -fPIC -Wall -pipe -fno-strict-aliasing -Wdeclaration-after-statement -Wwrite-strings -Wtype-limits -Wpointer-arith -Wall -g -O2 -o main.o main.c ../../tools/winegcc/winegcc -b x86_64-linux-gnu -m32 -B../../tools/winebuild -shared ./acledit.spec main.o -o acledit.dll.so -lkernel32 ../../libs/port/libwine_port.a -L/build/buildd/wine-1.1.15~winehq0~ubuntu~8.10/debian/libtmp sh: x86_64-linux-gnu-as: not found winebuild: x86_64-linux-gnu-as --32 failed with status 32512 winegcc: ../../tools/winebuild/winebuild failed Looks like a broken build environment to me. > I'm going to sleep now, but hopefully I'll figure this out Valentine's > day. In the meanwhile, 1.1.15 won't be available in Ubuntu packages > just yet. > > Thanks, > Scott Ritchie

Feb. 14, 2009