https://bugs.winehq.org/show_bug.cgi?id=42412
Bug ID: 42412 Summary: WinUAE 3.4.0: Networking over Slirp in AmigaOS 4.1 works bad. Product: Wine Version: 2.0 Hardware: x86 OS: Mac OS X Status: UNCONFIRMED Severity: normal Priority: P2 Component: winsock Assignee: wine-bugs@winehq.org Reporter: bobben@wigilius.se
When using WinUAE (both version 3.3.0 and 3.4.0) with slirp networking (only way when running AmigaOS 4.1), network transfers gets damaged (md5 and file size doesn't match) whether it's downloaded from internet or from local file server. This happens both on 32bit and 64bit Wine and WinUAE. Changing the MTU value on the AmigaOS side doesn't help either.
I suspect this has something to do with the issue:
fixme:winsock:set_dont_fragment IP_DONTFRAGMENT for IPv4 not supported in this platform.
The author of WinUAE also suspects this is the problem: "IP_DONTFRAG (or IP_MTU_DISCOVER and friends) are not defined in some platform specific file"
Currently using Wine 2.0-staging and macOS 10.11.6
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #1 from Bruno Jesus 00cpxxx@gmail.com --- It is unlikely that the problem is IP_DONTFRAGMENT. The app may not even be using it. Can you attach a +winsock log when doing the operation? Does this work in Windows? Like Windows->Windows, Windows->Wine, Wine->Windows, Wine->Wine.
Read more about how to get a log at https://wiki.winehq.org/FAQ#get_log
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #2 from bobben@wigilius.se --- Created attachment 57228 --> https://bugs.winehq.org/attachment.cgi?id=57228 Wine log with +winsock
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #3 from bobben@wigilius.se --- Yes, when running WinUAE 3.4.0 under Windows XP in Parallels Desktop 12, the exact same config and setup with AmigaOS 4.1 works fine.
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #4 from bobben@wigilius.se --- Forgot to add, that the log output with +winsock is from downloading a 1.2Mb file from my local file server over FTP.
The downloaded file ended up at 1230701 bytes, with the original file being 1230727 bytes, that is with the default MTU of 1500. Lowering the MTU will eventually get me to one byte from the original size, but never spot on.
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #5 from Bruno Jesus 00cpxxx@gmail.com --- It is possible to see some TCP connections, it looks like a FTP server due to port 21. The only socket options being used are SO_REUSEADDR and SO_OOBINLINE. The fourth connection is the data connection that really receives a lot of data from the server, there are no errors in the log unfortunately, everything seems to be smooth.
I believe WinUAE may have its own debug log (according to the source code), can you attach that here as well? I don't know how to enable it but it seem to exist.
Do you have a different software to test from inside Amiga OS? Like a web browser?
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #6 from bobben@wigilius.se --- Created attachment 57229 --> https://bugs.winehq.org/attachment.cgi?id=57229 Log from WinUAE with -slirplog 2
Downloading from local file server over FTP.
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #7 from bobben@wigilius.se --- Created attachment 57230 --> https://bugs.winehq.org/attachment.cgi?id=57230 Log from WinUAE with -a2065log
Download file from local file server over FTP.
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #8 from bobben@wigilius.se --- Created attachment 57231 --> https://bugs.winehq.org/attachment.cgi?id=57231 Log from WinUAE with -a2065log in Parallels Desktop 12 (Win XP).
File download from local file server over FTP, this time no problems.
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #9 from bobben@wigilius.se --- Discussion with WinUAE author:
http://eab.abime.net/showthread.php?t=85817
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #10 from Bruno Jesus 00cpxxx@gmail.com --- Thanks for the log, the one in comment 6 was what I was expecting. This is my only theory (aka wild guess) extracted from the slirp log+wine log+WinUAE sources:
WinUAE is caching all data read from the outside socket and delivering it to Amiga, but when the outside connection is closed WinUAE breaks the Amiga connection too instead of feeding it with the remaining cache before closing the Amiga connection. That means the final bytes of any transmission may randomly fail to be received depending on how fast the external connection is receiving and how fast Amiga is reading.
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #11 from bobben@wigilius.se --- Tested downloading with web browser (previous was FTP specific software) from local file server, this time the file size matches but not md5, and unarchiver in AmigaOS says the file is corrupted. Same if I download something from the internet.
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #12 from Bruno Jesus 00cpxxx@gmail.com --- That would also mean out of order delivery, which I don't think it is possible in Wine. Can you try Wine instead of Wine-Staging?
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #13 from bobben@wigilius.se --- Can't find any Wine other than developer or staging pre packaged for Mac OS.
https://bugs.winehq.org/show_bug.cgi?id=42412
fjfrackiewicz@gmail.com changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |fjfrackiewicz@gmail.com
--- Comment #14 from fjfrackiewicz@gmail.com --- (In reply to bobben from comment #13)
Can't find any Wine other than developer or staging pre packaged for Mac OS.
Wine-development is the one you want to test :)
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #15 from bobben@wigilius.se --- Created attachment 57234 --> https://bugs.winehq.org/attachment.cgi?id=57234 Wine (devel) log with +winsock
Downloaded a file (about 8Mb) from aminet.net with web browser, which is reported corrupt by the unarchiver in AmigaOS.
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #16 from bobben@wigilius.se --- Response from WinUAE developer:
http://eab.abime.net/showpost.php?p=1139950&postcount=29
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #17 from bobben@wigilius.se --- Have also tried Wine Devel 1.9.24, same issue there.
Can also mention that it seems to be working fine under Linux from what I have heard (haven't been able to test this myself), so seems to be MacOS specific currently.
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #18 from bobben@wigilius.se --- Have now tested in Linux and it doesn't work there either.
Testbed: Acer Travelmate 3043 Ubuntu Linux 14.04 64-bit
Tested with both wine/wine64 and WinUAE/WinUAE64 but no success.
https://bugs.winehq.org/show_bug.cgi?id=42412
bobben@wigilius.se changed:
What |Removed |Added ---------------------------------------------------------------------------- Depends on| |42432
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #19 from bobben@wigilius.se --- Created attachment 57341 --> https://bugs.winehq.org/attachment.cgi?id=57341 Wireshark capture in Windows and tcpdump inside AmigaOS (default mtu)
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #20 from bobben@wigilius.se --- Created attachment 57342 --> https://bugs.winehq.org/attachment.cgi?id=57342 Wireshark capture in Mac and tcpdump inside AmigaOS (mtu 1488)
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #21 from bobben@wigilius.se --- Created attachment 57343 --> https://bugs.winehq.org/attachment.cgi?id=57343 Wireshark capture in Mac and tcpdump inside AmigaOS (mtu 1500)
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #22 from bobben@wigilius.se --- Made some more captures this time with Wireshark and tcpdump inside Amiga.
The capture from Windows is with default MTU (1500) inside AmigaOS and downloading a 500kb file from local file server over FTP, which gives matching file size and md5 sum.
The capture from Mac (Wine 2.0 Devel) and tcpdump (in AmigaOS) is with mtu 1488 and same 500kb file over ftp, which gives matching file size and md5 sum.
The other from Mac (Wine 2.0 Devel) and tcpdump (in AmigaOS) is with default mtu 1500 and same 500kb file over ftp, which gives matching file size but wrong md5 sum.
So, mtu 1488 seems to work OK in Wine Mac, although painfully slow - about 17Kb/sec
All captures is with WinUAE 3.4 32bit and 32bit Wine on Mac.
https://bugs.winehq.org/show_bug.cgi?id=42412
bobben@wigilius.se changed:
What |Removed |Added ---------------------------------------------------------------------------- Attachment #57341|Wireshark capture in |Wireshark capture in description|Windows and tcpdump inside |Windows and tcpdump inside |AmigaOS (default mtu) |AmigaOS (mtu 1500)
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #23 from Bruno Jesus 00cpxxx@gmail.com --- If you compare both logs (1488 and 1500 mtu) you will see that they are nearly identical, most reads are the same size (1448 bytes).
If then you select the FTP download connection after the RETR for the file and select menu Analyze -> Follow -> TCP stream on both connections and then save both files as RAW and compare them the result will be 2 identical files. I also extracted them (since it is a LHA file) and SHA1 sum both extracted file contents (5 files), all matches. I double checked that I compared the OSX files.
The tcpdump logs are less informative for me, the weird part is that the 1500 byte MTU seems to be really respected as all reads are 1500 bytes, but the 1488 MTU works in packets of 770 bytes, so the log size is doubled. I don't know where this 770 number comes from.
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #24 from bobben@wigilius.se --- Created attachment 57361 --> https://bugs.winehq.org/attachment.cgi?id=57361 tcpdump with -eXXvvv from inside AmigaOS
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #25 from bobben@wigilius.se --- Just discovered something weird, if I run "tcpdump -eXXvvv > ram:test.txt" in AmigaOS I get matching md5 (despite mtu 1500) on files downloaded from local file server over ftp and as soon as I stop tcpdumping and download the same file again, md5 no longer matches... Attached the tcpdump above.
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #26 from Bruno Jesus 00cpxxx@gmail.com --- Are you using the ftp command line utility? What is its version? If you are lucky enough and it is this package [1] maybe you are hitting an app bug that was fixed some days ago.
[1] http://os4depot.net/?function=showfile&file=network/ftp/download.lha
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #27 from bobben@wigilius.se --- No, I'm using IBrowse 2.4 that is delivered with AmigaOS 4.1.
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #28 from bobben@wigilius.se --- Also tried downloading from internet, in this case aminet.net with same results, so not FTP specific.
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #29 from bobben@wigilius.se --- Any possibility to add more debug output in Wine for the networking, to make it easier to track down where the issue is? By that I mean adding debug output in the source code for an upcoming release...
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #30 from Bruno Jesus 00cpxxx@gmail.com --- If you are able to compile wine I can certainly add full data dumps from recv functions. But that sort of patch is not valuable to be streamlined so it can't go in a release (for example 2.3).
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #31 from bobben@wigilius.se --- Yes, I should be able to compile Wine.
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #32 from bobben@wigilius.se --- Successfully built Wine and test run with WinUAE.
So, what's the next step? A patch from you, to apply to the source or?
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #33 from Bruno Jesus 00cpxxx@gmail.com --- Created attachment 57419 --> https://bugs.winehq.org/attachment.cgi?id=57419 debug test
Please transfer the same lha file from previous tests using the FTP and this patch.
To apply the patch if you used git to clone Wine you can cd into the wine root dir and "git apply /path/to/test.txt" and then make.
Compress and attach there the resulting set of /tmp/sock-XXXX.txt files.
A wireshark capture at the same time would be a plus.
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #34 from bobben@wigilius.se --- Created attachment 57438 --> https://bugs.winehq.org/attachment.cgi?id=57438 Logs from both mtu 1488 and 1500
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #35 from bobben@wigilius.se --- Logs and captures added from self built Wine 2.0 with patch.
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #36 from Bruno Jesus 00cpxxx@gmail.com --- Thanks again for the log once again. The contents of the sockets on both MTU configurations match as expected.
SHA1 of the files and size: ac94564c3cb9ab7923bb6c2fd1ad8373289dfedd 1488/sock-03fc.txt 517331 bytes ac94564c3cb9ab7923bb6c2fd1ad8373289dfedd 1500/sock-03ec.txt 517331 bytes
The first 146 lines in the socket are the FTP LIST result. If you remove these lines save the file again and extract:
------------------------------------------------------ $ 7z x sock-03ec.txt ... Extracting archive: sock-03ec.txt -- Path = sock-03ec.txt Type = Lzh Physical Size = 506501
Everything is Ok
Files: 5 Size: 997616 Compressed: 506501 ------------------------------------------------------
In other words, the flow and order from the network -> host os -> wine -> winuae is as expected. The problem can be either in winuae -> amiga os -> application.
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #37 from bobben@wigilius.se --- Have also tested with AmigaOS 3.9 (68k) with MiamiDX as TCP/IP stack (on the Amiga) and emulated A2065 network card with Slirp and it has the same problem with corrupted files that are downloaded, either from internet or local fileserver.
But when I use bsdsocket.library emulation (which doesn't use Slirp) problem goes away, but can't use that for AmigaOS 4. So that pretty much rules out all on the Amiga side (OS, TCP/IP stack and download software).
Then why does it work in Wine under Linux? Is there something else that can screw things up when it reassembles the TCP packets, that is different on Mac Wine, like how it handles threads or something else... ?
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #38 from Bruno Jesus 00cpxxx@gmail.com --- Certainly this is a mysterious bug. I currently have no more ideas Wine related.
You could try other emulator to eliminate the WinUAE variable. http://alternativeto.net/software/winuae/
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #39 from bobben@wigilius.se --- Yes, I could try another, but WinUAE is the best and fastest and is the most developed of them all, so I really like to be able to use it.
Did a small capture with tcpdump in AmigaOS 3.9 with RoadShow TCP/IP stack of downloading a small file from local fileserver, same like before.
This dump can be opened in Wireshark and actually says tcp out of order in a couple of places... Don't know if it's of any use to you, but trying anyway :-)
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #40 from bobben@wigilius.se --- Created attachment 57499 --> https://bugs.winehq.org/attachment.cgi?id=57499 Tcpdump for Wireshark from AmigaOS 3.9
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #41 from Bruno Jesus 00cpxxx@gmail.com --- I'm not saying you should dump WinUAE, it is clearly the most talked about Amiga emulator. I'm saying that you could try alternatives to check if the problem is also present in other emulators.
The magic of TCP is that it takes care of a lot of common problems in data transmission like lost packages, out of order and congestion control. So it is very common in Wireshark to see black lines of out of order or resent packages.
A different approach would be to write a server application that sends a well-known set of data (eg 2048 bytes of each alphabet character) and checking if a client inside Amiga receives the data correctly. If the data is received correctly the problem is the FTP application.
Another approach would be to run Wireshark inside Amiga.
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #42 from bobben@wigilius.se --- Have managed to get BasiliskII for Windows to run in Wine and that also uses Slirp, actually the Slirp code in WinUAE is from BasiliskII.
And I can download both from local file server (FTP) and internet and the files do not seem to get corrupted, since StuffitExpander can unpack them without errors.
So, it starts to seem more and more likely that the problem is in WinUAE, that only gets triggered in Wine for Mac...
It hasn't gotten something to do with pcap support in Wine? Because this can be seen in winuaebootlog.txt: uaenet: npcap/winpcap not installed (packet.dll) If slirp needs it somehow and Mac Wine version doesn't have same implementation completeness as Wine Linux?
https://bugs.winehq.org/show_bug.cgi?id=42412 Bug 42412 depends on bug 42432, which changed state.
Bug 42432 Summary: WinUAE 3.4.0: Networking over Slirp in AmigaOS 4.1 works bad (Linux). https://bugs.winehq.org/show_bug.cgi?id=42432
What |Removed |Added ---------------------------------------------------------------------------- Status|UNCONFIRMED |RESOLVED Resolution|--- |WORKSFORME
https://bugs.winehq.org/show_bug.cgi?id=42412
Damjan Jovanovic damjan.jov@gmail.com changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |damjan.jov@gmail.com
--- Comment #43 from Damjan Jovanovic damjan.jov@gmail.com --- Hi, are you still interested in having this bug fixed?
I tried running WinUAE with Aros Vision to try and reproduce it, but WinUAE seems very difficult to configure, and it keeps crashing on startup.
If you can reproduce this bug in Aros, please provide the version of Wine used, the version of WinUAE, the version of Aros, and the configuration settings used (the .uae file), and I'll try to investigate further.
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #44 from bobben@wigilius.se --- Hi!
Yes, I would like to get it fixed!
I am able to reproduce it in Aros Vision as well. Will require some extra fiddling since Aros own network software doesn't work, so Miami has to be installed and to be able to install it, the original installer from OS 3.x needs to be copied over the Aros version in the C dir since the Aros installer can't parse the install script properly. Then ethernet.lha needs to be downloaded from Aminet for the A2065 network card.
Currently using Wine 4.15 Dev (32-bit) for Mac, WinUAE 4.2.0 and Aros Vision from http://download.aros-platform.de/Aros_Vision.zip which should be version Sep 5 2018 SVN 55425.
https://bugs.winehq.org/show_bug.cgi?id=42412
--- Comment #45 from bobben@wigilius.se --- Created attachment 65232 --> https://bugs.winehq.org/attachment.cgi?id=65232 WinUAE Aros Vision configuration.
https://bugs.winehq.org/show_bug.cgi?id=42412
bobben@wigilius.se changed:
What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |NOTOURBUG Status|UNCONFIRMED |RESOLVED
--- Comment #46 from bobben@wigilius.se --- Some new information on this issue, seems it's an issue with libslirp specifically on macOS.
https://gitlab.freedesktop.org/slirp/libslirp/-/issues/35
and
https://eab.abime.net/showthread.php?t=108962
So the issue seems to be wholly in WinUAE's slirp implementation and has nothing to do with Wine, so I think this issue can be closed.
https://bugs.winehq.org/show_bug.cgi?id=42412
Gijs Vermeulen gijsvrm@gmail.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Resolution|NOTOURBUG |INVALID Status|RESOLVED |CLOSED
--- Comment #47 from Gijs Vermeulen gijsvrm@gmail.com --- Changing to INVALID, since it isn't a bug in a library that wine directly depends on.
This was fixed in libslirp 4.5.0 (released May 18th 2021).
Closing.
-
wine-bugs＠winehq.org -
WineHQ Bugzilla