[Bug 45847] New: Warframe installer update/ download gets stuck due to corruption on large files in 'Bulk Download' mode (tries over and over again)
https://bugs.winehq.org/show_bug.cgi?id=45847 Bug ID: 45847 Summary: Warframe installer update/download gets stuck due to corruption on large files in 'Bulk Download' mode (tries over and over again) Product: Wine Version: 3.16 Hardware: x86-64 OS: Linux Status: NEW Severity: normal Priority: P2 Component: wininet Assignee: wine-bugs(a)winehq.org Reporter: focht(a)gmx.net Distribution: --- Hello folks, cloned off from bug 45734 There is troll who has no idea how Bugzilla works. I made the mistake to comment there. Partial copy/pasta of my initial analysis from https://bugs.winehq.org/show_bug.cgi?id=45734#c12 Prerequisites: * work around bug 38960 ("Multiple applications need 'kernel32.SetFileCompletionNotificationModes' implementation (Steam 'steamwebhelper.exe' and Blizzard Battle.net desktop app CEF v3.2+/Chromium v44+ crash in Win7 mode, Sony Xperia Companion 1.7.x)" -> 'winetricks winxp' or use Wine-Staging if you need 'Windows 7' mode for whatever reasons * work around bug 37797 ("Warframe updater/installer crashes (CEF3 with Blink/Google Chrome and V8 checks requested font facename, doesn't like Liberation replacement, Arial font)") -> dupe of bug 32342 ("Multiple applications and games using 'Webkit' or 'Blink' web browser engine crash on startup ('Arial' and 'Times New Roman' font face name validation)") -> 'winetricks -q corefonts' or use Wine-Staging --- snip --- $ find .wine -iname *.log .wine/winetricks.log .wine/drive_c/windows/DirectX.log .wine/drive_c/windows/logs/DirectX.log .wine/drive_c/users/focht/Local Settings/Application Data/Warframe/Launcher.log --- snip --- 'Launcher.log' is the log file one needs to check for download problems/errors. It shows each file/chunk downloaded. There seems to be a problem with very large files (archives) when 'Bulk Download' mode is used. For example there is always corruption happening in some chunks for the file 'F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma' (14 GB). Since it's streaming LZMA it gets detected (see "Bad LZMA stream" in launcher log) and re-downloaded over and over again. --- snip --- ... Queuing Z:\home\focht\Downloaded\Public\Cache.Windows\F.TextureDx9.cache [7909D15B39359A27EDEF21DEB9D86D3B] Queuing Z:\home\focht\Downloaded\Public\Cache.Windows\F.TextureDx9.toc [DCBE1867172A86A365910B6CB36BFC4B] ... HTTP GET /Cache.Windows/F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma GET //Cache.Windows/B.Misc.cache.17C3FB9222153CF642694D33955737BD.lzma HTTP/1.1 Host: content.warframe.com Connection: Keep-Alive HTTP/1.1 200 OK Date: Sun, 16 Sep 2018 11:40:51 GMT Connection: Keep-Alive Accept-Ranges: bytes Cache-Control: public, max-age=86400 Content-Length: 2035849776 Content-Type: chemical/x-cache Last-Modified: Wed, 01 Aug 2018 20:21:44 GMT ETag: "79589a30-572657175ce04" X-HW: 1537098051.dop014.fr8.t,1537098051.cds035.fr8.c GET //Cache.Windows/F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma HTTP/1.1 Host: content.warframe.com Connection: Keep-Alive HTTP/1.1 200 OK Date: Sun, 16 Sep 2018 11:40:51 GMT Connection: Keep-Alive Accept-Ranges: bytes Cache-Control: public, max-age=86400 Content-Length: 14112683568 Content-Type: chemical/x-cache Last-Modified: Wed, 01 Aug 2018 20:24:53 GMT ETag: "3492e7630-572657cb775b4" X-HW: 1537098051.dop002.fr8.t,1537098051.cds003.fr8.c Downloading @ 6 MB/s Downloading @ 6 MB/s ... Downloading @ 6 MB/s Bad LZMA stream in //Cache.Windows/F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma HTTP GET /Cache.Windows/F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma GET //Cache.Windows/F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma HTTP/1.0 Host: content.warframe.com Pragma: no-cache Connection: Keep-Alive HTTP/1.0 200 OK LzmaDec_Allocate failed for //Cache.Windows/F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma HTTP GET /Cache.Windows/F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma GET //Cache.Windows/F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma HTTP/1.1 Host: content.warframe.com Pragma: no-cache Connection: Keep-Alive HTTP/1.1 200 OK Date: Sun, 16 Sep 2018 11:54:01 GMT Connection: Keep-Alive Accept-Ranges: bytes Cache-Control: public, max-age=86400 Content-Length: 14112683568 Content-Type: chemical/x-cache Last-Modified: Wed, 01 Aug 2018 20:24:53 GMT ETag: "3492e7630-572657cb775b4" X-HW: 1537098841.dop017.fr8.t,1537098841.cds003.fr8.c Downloading @ 6 MB/s Downloading @ 6 MB/s ... Downloading @ 6 MB/s Bad LZMA stream in //Cache.Windows/F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma HTTP GET /Cache.Windows/F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma GET //Cache.Windows/F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma HTTP/1.0 Host: content.warframe.com Pragma: no-cache Connection: Keep-Alive HTTP/1.0 200 OK LzmaDec_Allocate failed for //Cache.Windows/F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma HTTP GET /Cache.Windows/F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma GET //Cache.Windows/F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma HTTP/1.1 Host: content.warframe.com Pragma: no-cache Connection: Keep-Alive HTTP/1.1 200 OK Date: Sun, 16 Sep 2018 12:05:44 GMT Connection: Keep-Alive Accept-Ranges: bytes Cache-Control: public, max-age=86400 Content-Length: 14112683568 Content-Type: chemical/x-cache Last-Modified: Wed, 01 Aug 2018 20:24:53 GMT ETag: "3492e7630-572657cb775b4" X-HW: 1537099544.dop009.fr8.t,1537099544.cds003.fr8.c Downloading @ 6 MB/s Downloading @ 6 MB/s ... Downloading @ 6 MB/s Bad LZMA stream in //Cache.Windows/F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma Downloaded 11,630,780,172 bytes total (25,743,463,740 expected) ~ 6,349,930 B/s Finished Binaries1 (1831.7 seconds) ... --- snip --- To work around, uncheck the 'Bulk download' mode in settings. Interestingly this mode is not available in 32-bit WINEPREFIX. Both, '64-bit Mode' and 'Bulk Download' are automatically disabled in settings there. The downloads are ordinary webserver/webdav file requests via 'wininet'. If you start the launcher as follows you will see all of them (lots of spam though): --- snip --- $ WINEDEBUG=+loaddll,+process,+wininet wine msiexec -i Warframe.msi --- snip --- All the files can be downloaded without Warframe installer/launcher by tracing the installer on +wininet debug channel. Their content is delivered as LZMA compressed files/streams. Get list of all files to be downloaded: --- snip --- $ wget -qO- /dev/null content.warframe.com//origin/310F099E/index.txt.lzma | lzmadec ... /Cache.Windows/F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma,14112683568 --- snip --- NOTE: All paths are relative to your Warframe download folder. It depends on your selection when your started the updater first time. I selected "c:\\warframe" in my case. --- snip --- $ pwd /home/focht/.wine/drive_c/warframe/Downloaded/Public --- snip --- Download one specific file via 'wget' from 'content.warframe.com' using the file path from list: --- snip --- $ wget content.warframe.com/Cache.Windows/F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma --2018-09-16 14:58:15-- http://content.warframe.com/Cache.Windows/F.TextureDx9.cache.7909D15B39359A2... Resolving content.warframe.com (content.warframe.com)... 205.185.216.10, 205.185.216.42 Connecting to content.warframe.com (content.warframe.com)|205.185.216.10|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 14112683568 (13G) [chemical/x-cache] Saving to: ‘F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma’ F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D 100%[==========================================================================================================>] 13.14G 6.07MB/s in 39m 42s 2018-09-16 15:37:57 (5.65 MB/s) - ‘F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma’ saved [14112683568/14112683568] $ lzmainfo F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma Uncompressed size: 15749 MB (16514030011 bytes) Dictionary size: 64 MB (2^26 bytes) Literal context bits (lc): 3 Literal pos bits (lp): 0 Number of pos bits (pb): 2 $ xz --format=lzma --decompress F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B.lzma $ md5sum F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B 7909d15b39359a27edef21deb9d86d3b F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B $ cp F.TextureDx9.cache.7909D15B39359A27EDEF21DEB9D86D3B ~/.wine/drive_c/warframe/Downloaded/Public/Cache.Windows/F.TextureDx9.cache --- snip --- Essentially with a few script lines you could write your own Warframe updater/downloader ;-) --- snip --- # retrieve all files for 'Cache.Windows' folder in Warframe download cache $ wget -qO- /dev/null content.warframe.com//origin/310F099E/index.txt.lzma | lzmadec | sed -ne 's/^\(\/Cache.Windows.*\),.*/content.warframe.com\1/p' | xargs wget -c # uncompress downloaded files (keep originals) $ find . -iname "*.lzma" -exec xz --format=lzma --decompress --keep {} \; # copy all files that have the md5sum pattern extension to target folder (while dropping the md5sum extension) $ find . -regextype sed -regex ".*.[A-F0-9]\{31\}$" -exec bash -c 'cp "$0" ~/".wine/drive_c/warframe/Downloaded/Public/Cache.Windows/${0%.*}"' {} \; --- snip --- This looks like 'wininet' related. Unfortunately 'winetricks -q wininet' provides a very old version of 'wininet.dll' which can't be used due to Win7+ features (Warframe will refuse). $ sha1sum Warframe.msi cd9b81c88d0b2888772e1cab570246d5b5b29131 Warframe.msi $ du -sh Warframe.msi 40M Warframe.msi $ wine --version wine-3.16 Regards -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=45847 Anastasius Focht <focht(a)gmx.net> changed: What |Removed |Added ---------------------------------------------------------------------------- URL| |https://www.warframe.com/do | |wnload Keywords| |download, Installer -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=45847 Fabian Maurer <dark.shadow4(a)web.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dark.shadow4(a)web.de --- Comment #1 from Fabian Maurer <dark.shadow4(a)web.de> --- Did you already test getting a wininet from win7/win10 and using that? That should (in theory) work when it's a wininet issue. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=45847 zzzzzyzz(a)hacari.org changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |zzzzzyzz(a)hacari.org -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=45847 Matteo Bruni <matteo.mystral(a)gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |swojskichlopak(a)wp.pl --- Comment #2 from Matteo Bruni <matteo.mystral(a)gmail.com> --- *** Bug 45734 has been marked as a duplicate of this bug. *** -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=45847 GloriousEggroll(a)gmail.com changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |GloriousEggroll(a)gmail.com --- Comment #3 from GloriousEggroll(a)gmail.com --- (In reply to Fabian Maurer from comment #1)
Did you already test getting a wininet from win7/win10 and using that? That should (in theory) work when it's a wininet issue.
using an official wininet from win7 or using wininet from winetricks will not work. Launcher crashes here when using native wininet: wine: Call from 0x7bc7e6a1 to unimplemented function urlmon.dll.414, aborting wine: Unimplemented function urlmon.dll.414 called at address 0x7bc7e6a1 (thread 006a), starting debugger... 0033:fixme:powermgnt:PowerCreateRequest (0x32ed1c): stub -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=45847 --- Comment #4 from Fabian Maurer <dark.shadow4(a)web.de> --- Maybe native urlmon as well...? -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=45847 --- Comment #5 from GloriousEggroll(a)gmail.com --- (In reply to Fabian Maurer from comment #4)
Maybe native urlmon as well...?
I tried that as well (common sense). No luck. I can post a log later today if need. Apologies for not mentioning that after the fact. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=45847 zefkerr <zefkerrigan(a)gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |zefkerrigan(a)gmail.com -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=45847 Anastasius Focht <focht(a)gmx.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Fixed by SHA1| |e07efbd9197a4628024dce9338b | |c3c947280662c URL|https://www.warframe.com/do |https://web.archive.org/web |wnload |/20200122094852/http://cont | |ent.warframe.com/dl/Warfram | |e.msi Resolution|--- |FIXED Status|NEW |RESOLVED --- Comment #6 from Anastasius Focht <focht(a)gmx.net> --- Hello folks, revisiting, this was fixed by commit https://source.winehq.org/git/wine.git/commitdiff/e07efbd9197a4628024dce9338... ("wininet: Support large downloads."), part of Wine 4.16 release. Thanks Daniel. Stable download link from Internet Archive: https://web.archive.org/web/20200122094852/http://content.warframe.com/dl/Wa... For testing the error case automatically, run the installer via 'wine msiexec -i Warframe.msi' and in another terminal: --- snip --- $ tail -f ~/.wine/drive_c/users/focht/Local\ Settings/Application\ Data/Warframe/Launcher.log | sed '/^Bad LZMA/ q' ; wineserver -k --- snip --- It pipes everything written to launcher log file into 'sed' and tells it to quit when the error pattern is seen. The installer is terminated as well to avoid resume. $ sha1sum Warframe.msi 9c957237c572e666dec89249a01c8313b79c892a Warframe.msi $ du -sh Warframe.msi 38M Warframe.msi $ wine --version wine-5.0 Regards -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=45847 Alexandre Julliard <julliard(a)winehq.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|RESOLVED |CLOSED --- Comment #7 from Alexandre Julliard <julliard(a)winehq.org> --- Closing bugs fixed in 5.1. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
participants (2)
-
wine-bugs＠winehq.org -
WineHQ Bugzilla