[Bug 46755] New: File paths with non-ASCII characters (umlauts) broken in Wine apps
https://bugs.winehq.org/show_bug.cgi?id=46755 Bug ID: 46755 Summary: File paths with non-ASCII characters (umlauts) broken in Wine apps Product: Wine Version: 4.2 Hardware: x86 OS: Linux Status: UNCONFIRMED Severity: normal Priority: P2 Component: -unknown Assignee: wine-bugs(a)winehq.org Reporter: debilerpc(a)web.de Distribution: --- Created attachment 63761 --> https://bugs.winehq.org/attachment.cgi?id=63761 locale This only started occuring after I updated wine to 4.2-1 My OS is Arch Linux, 64 bit. Noticed it in Foobar2000. Playlist that contain songs with file paths that contain German umlauts or other non-ASCII characters skip over those songs when trying to play them with an error like this:
Unable to open item for playback (bad exception): "E:\files\sonofevil\Music\Musik\Schönberg, Arnold\Boulez conducts Schönberg\CD 01\07 - 3 Pieces for Chamber Orchestra - II. Máige Viertel.flac"
This seems to be because to Wine this Item doesn't seem to exist. That's because when Wine accesses the file system, non-ASCII characters are automatically interpreted as something like this:
...\Schönberg, Arnold\...
As I found by browsing for that folder using the "wine explorer" file browser. In native Linux apps, the file paths work just fine on the other hand, with the directory being shown as "Schönberg, Arnold" everywhere. The problem only occurs in Wine apps. So when it tries to access the directory specified in the Foobar playlist as "Schönberg, Arnold" it concludes that this folder doesn't exist. Note that on the side of the app everything is seemingly fine. The non-ASCII characters are read and displayed correctly in Foobar, it's just when Foobar asks Wine whether the file exists, Wine gives the wrong response because whatever does the file accessing in Wine can't handle those characters. Solutions on the internet for problems like this tell you to fix the locale variables, but my locale settings seem to be perfectly fine. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=46755 debilerpc(a)web.de changed: What |Removed |Added ---------------------------------------------------------------------------- Hardware|x86 |x86-64 -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=46755 Daniil Zhilin <just4steam778(a)gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |just4steam778(a)gmail.com --- Comment #1 from Daniil Zhilin <just4steam778(a)gmail.com> --- The issue still exists on Wine 4.6-1. Files with paths containing Cyrillic or Japanese characters are also only available using paths converted to ISO 8859-1. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=46755 Olivier F. R. Dierick <o.dierick(a)piezo-forte.be> changed: What |Removed |Added ---------------------------------------------------------------------------- URL| |https://www.foobar2000.org/ | |download CC| |o.dierick(a)piezo-forte.be --- Comment #2 from Olivier F. R. Dierick <o.dierick(a)piezo-forte.be> --- Hello, foobar2000 v1.4.3 works fine for me with wine 4.6. $ du -b foobar2000_v1.4.3.exe 4452151 foobar2000_v1.4.3.exe $ sha1sum foobar2000_v1.4.3.exe de67cdd9b5bb46f4e6c7491f23501625c94594ee foobar2000_v1.4.3.exe My system is Debian 8 Jessie 64 bit with locale fr_BE.utf8 $ export | grep 'utf8' declare -x GDM_LANG="fr_BE.utf8" declare -x LANG="fr_BE.utf8" declare -x LC_MEASUREMENT="fr_BE.utf8" declare -x LC_MONETARY="fr_BE.utf8" declare -x LC_NUMERIC="fr_BE.utf8" declare -x LC_PAPER="fr_BE.utf8" declare -x LC_TIME="fr_BE.utf8" I copied a random MP3 file and renamed it to '/home/olivier/Musique/Schönberg, Arnold/Boulez conducts Schönberg/CD 01/07 - 3 Pieces for Chamber Orchestra - II. Máige Viertel.mp3' (Copy-pasted the pathnames from the bug to the linux file manager). /home is on a linux ext4 partition. Using a new 32 bit wineprefix with everything set to default values: - Installed foobar2000; - Launched foobar2000 from its directory; - Choose default appearance; - Clicked open in the File menu; - Browsed to the file on the z: drive; - Double-clicked to open the file; - foobar2000 added the file to the playlist and played it. Please, attach full terminal output. Read how to do so there: https://wiki.winehq.org/FAQ#How_can_I_get_a_debugging_log_.28a.k.a._terminal... Regards -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=46755 --- Comment #3 from Daniil Zhilin <just4steam778(a)gmail.com> --- Created attachment 64233 --> https://bugs.winehq.org/attachment.cgi?id=64233 Terminal output Sure, here you go. Are you sure the path you choose is correct? The issue is with non-ASCII paths not being read from old playlists and not being saved as Unicode in new playlists. I can still play the files fine after I've added them to the playlist directly. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=46755 --- Comment #4 from Olivier F. R. Dierick <o.dierick(a)piezo-forte.be> --- Hello, The issue is unclear and I'm doing with what I'm given. I need some clarifications to try and reproduce the issue. Following the last comment, I also tried this: - Saved a new playlist with the single file with an utf-8 filename I created earlier, restarted foobar2000 and then loaded the playlist back and play the file. - Loaded the playlist and save it under another name. All worked without issue for me. Is that failing for you? (In reply to debilerpc from comment #0)
This only started occuring after I updated wine to 4.2-1
Updated from which version? You should do a regression test if you can. Read how to do it there: https://wiki.winehq.org/Regression_Testing
Noticed it in Foobar2000. Playlist that contain songs with file paths that contain German umlauts or other non-ASCII characters skip over those songs when trying to play them with an error like this:
Unable to open item for playback (bad exception): "E:\files\sonofevil\Music\Musik\Schönberg, Arnold\Boulez conducts Schönberg\CD 01\07 - 3 Pieces for Chamber Orchestra - II. Máige Viertel.flac"
That error doesn't show in the terminal output. Does it appear in a dialog box?
This seems to be because to Wine this Item doesn't seem to exist. That's because when Wine accesses the file system, non-ASCII characters are automatically interpreted as something like this:
...\Schönberg, Arnold\...
As I found by browsing for that folder using the "wine explorer" file browser. In native Linux apps, the file paths work just fine on the other hand, with the directory being shown as "Schönberg, Arnold" everywhere. The problem only occurs in Wine apps.
I'm not sure about this. Are you saying that wine explorer display wrong characters?
Note that on the side of the app everything is seemingly fine. The non-ASCII characters are read and displayed correctly in Foobar, it's just when Foobar asks Wine whether the file exists, Wine gives the wrong response because whatever does the file accessing in Wine can't handle those characters.
I fail to see when exactly the issue happens. Which menu/button command sequence in foobar2000 does fail? Give exact steps from starting foobar2000 to the error message (menus/buttons clicked). (In reply to Daniil Zhilin from comment #1)
The issue still exists on Wine 4.6-1. Files with paths containing Cyrillic or Japanese characters are also only available using paths converted to ISO 8859-1.
What does that mean: Paths converted to ISO 8859-1? How do you convert them? Can you tell the filesystem type where the audio files are stored? (In reply to Daniil Zhilin from comment #3)
Are you sure the path you choose is correct? The issue is with non-ASCII paths not being read from old playlists and not being saved as Unicode in new playlists. I can still play the files fine after I've added them to the playlist directly.
'Read from old playlists and not being saved as Unicode in new playlists'. Can you explain the exact steps? How were the old playlists created? Was it under a different OS or Wine version? It is possible that the playlists were created on a ISO-8859-1 system and foobar2000 stored the filenames as-is, though I don't know why it would have worked before and then stopped working. Do any of you have a short playlist that exhibits the issue that you can attach to this bug? Please, attach a WINEDEBUG=+file log too, as it might help understand what happens when the application tries to access the files. (You may need to compress the log if it's big >5MB) Also, what are your versions of foobar2000? Regards -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=46755 --- Comment #5 from Daniil Zhilin <just4steam778(a)gmail.com> --- It's quite simple really. The path in the playlist you've saved should look something like "Schönberg, Arnold\Boulez conducts Schönberg\CD 01\07 - 3 Pieces for Chamber Orchestra - II. Máige Viertel.flac". You can also see these incorrectly encoded names in the Wine Explorer. The old playlists we're having an issue with were created on an older Wine version. I've tested multiple foobar2000 versions including the latest one (1.4.3). -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=46755 --- Comment #6 from Olivier F. R. Dierick <o.dierick(a)piezo-forte.be> --- Created attachment 64238 --> https://bugs.winehq.org/attachment.cgi?id=64238 foobar2000 & wine explorer utf-8 path display Hello, I see no obvious issue with the utf-8 names in foobar2000 or wine explorer. $ file foobar2000_test.fpl foobar2000_test.fpl: data $ strings -e S foobar2000_test.fpl ��<wB�,;� file://Z:\home\olivier\Musique\Schönberg, Arnold\Boulez conducts Schönberg\CD 01\07 - 3 Pieces for Chamber Orchestra - II. Máige Viertel.mp3 bitrate codec encoding lossy channels samplerate 44100 mp3_stereo_mode joint stereo codec_profile tagtype id3v1 ��XL_@ This command ("strings") extracts utf-8 encoded ("-e S") strings from the binary file (.fpl playlist). The playlist I created returns proper utf-8 characters in the paths and filenames. Can you provide screen shots for comparison and the result of this command with one of your old playlists? Regards -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=46755 --- Comment #7 from Daniil Zhilin <just4steam778(a)gmail.com> --- Created attachment 64239 --> https://bugs.winehq.org/attachment.cgi?id=64239 WINEDEBUG=+file output Well, OK, hopefully there's not a ton of compromising data there. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=46755 --- Comment #8 from Daniil Zhilin <just4steam778(a)gmail.com> --- Created attachment 64240 --> https://bugs.winehq.org/attachment.cgi?id=64240 foobar2000 & wine explorer utf-8 path display 2 I'm using an M3U file for simplicity. $ file 1.m3u8 1.m3u8: M3U playlist, UTF-8 Unicode text $ strings -e S 1.m3u8 #EXTM3U #EXTINF:245,Кино - Время есть, а денег нет Кино (Виктор Цой) - Дискография/1982 - 45/01 - Время есть, а денег нет.mp3 #EXTINF:206,Кино - Просто хочешь ты знать Кино (Виктор Цой) - Дискография/1982 - 45/02 - Просто хочешь ты знать.mp3 #EXTINF:174,Кино - Аллюминиевые огурцы Кино (Виктор Цой) - Дискография/1982 - 45/03 - Аллюминиевые огурцы.mp3 #EXTINF:190,Кино - Солнечные дни Кино (Виктор Цой) - Дискография/1982 - 45/04 - Солнечные дни.mp3 #EXTINF:191,Кино - Бездельник Кино (Виктор Цой) - Дискография/1982 - 45/05 - Бездельник.mp3 #EXTINF:184,Кино - Бездельник 2 Кино (Виктор Цой) - Дискография/1982 - 45/06 - Бездельник 2.mp3 #EXTINF:154,Кино - Электричка Кино (Виктор Цой) - Дискография/1982 - 45/07 - Электричка.mp3 #EXTINF:163,Кино - Восьмиклассница Кино (Виктор Цой) - Дискография/1982 - 45/08 - Восьмиклассница.mp3 #EXTINF:264,Кино - Мои друзья Кино (Виктор Цой) - Дискография/1982 - 45/09 - Мои друзья.mp3 #EXTINF:92,Кино - Ситар играл Кино (Виктор Цой) - Дискография/1982 - 45/10 - Ситар играл.mp3 #EXTINF:101,Кино - Дерево Кино (Виктор Цой) - Дискография/1982 - 45/11 - Дерево.mp3 #EXTINF:218,Кино - Когда-то ты был битником Кино (Виктор Цой) - Дискография/1982 - 45/12 - Когда-то ты был битником.mp3 #EXTINF:173,Кино - На кухне Кино (Виктор Цой) - Дискография/1982 - 45/13 - На кухне.mp3 #EXTINF:188,Кино - Я - асфальт (bonus track) Кино (Виктор Цой) - Дискография/1982 - 45/14 - Я - асфальт (bonus track).mp3 The weird folder names in Wine Explorer are supposed to be Cyrillic. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=46755 --- Comment #9 from Olivier F. R. Dierick <o.dierick(a)piezo-forte.be> --- Hello, (In reply to Daniil Zhilin from comment #7)
Created attachment 64239 [details] WINEDEBUG=+file output
I found this in your log: --- output --- (...) 0033:trace:file:CreateFileW L"\\\\?\\Z:\\home\\daniil\\real_home\\Music\\Sch\00c3\00b6nberg, Arnold\\Boulez conducts Sch\00c3\00b6nberg\\CD 01\\07 - 3 Pieces for Chamber Orchestra - II. M\00c3\00a1ige Viertel.flac" GENERIC_READ FILE_SHARE_READ creation 3 attributes 0x40000000 0033:trace:file:RtlDosPathNameToNtPathName_U_WithStatus (L"\\\\?\\Z:\\home\\daniil\\real_home\\Music\\Sch\00c3\00b6nberg, Arnold\\Boulez conducts Sch\00c3\00b6nberg\\CD 01\\07 - 3 Pieces for Chamber Orchestra - II. M\00c3\00a1ige Viertel.flac",0xbc8f5b4,(nil),(nil)) (...) 0033:trace:file:wine_nt_to_unix_file_name L"\\??\\Z:\\home\\daniil\\real_home\\Music\\Sch\00c3\00b6nberg, Arnold\\Boulez conducts Sch\00c3\00b6nberg\\CD 01\\07 - 3 Pieces for Chamber Orchestra - II. M\00c3\00a1ige Viertel.flac" -> "/home/daniil/.wine/dosdevices/z:/home/daniil/real_home/Music/Sch\xc3\xb6nberg, Arnold/Boulez conducts Sch\xc3\xb6nberg/CD 01/07 - 3 Pieces for Chamber Orchestra - II. M\xc3\xa1ige Viertel.flac" (...) --- end output --- Mine is like this: --- output --- (...) 0036:trace:file:CreateFileW L"\\\\?\\Z:\\home\\olivier\\Musique\\Sch\00f6nberg, Arnold\\Boulez conducts Sch\00f6nberg\\CD 01\\07 - 3 Pieces for Chamber Orchestra - II. M\00e1ige Viertel.mp3" GENERIC_READ FILE_SHARE_READ creation 3 attributes 0x40000000 0036:trace:file:RtlDosPathNameToNtPathName_U_WithStatus (L"\\\\?\\Z:\\home\\olivier\\Musique\\Sch\00f6nberg, Arnold\\Boulez conducts Sch\00f6nberg\\CD 01\\07 - 3 Pieces for Chamber Orchestra - II. M\00e1ige Viertel.mp3",0x1d8f624,(nil),(nil)) 0036:trace:file:wine_nt_to_unix_file_name L"\\??\\Z:\\home\\olivier\\Musique\\Sch\00f6nberg, Arnold\\Boulez conducts Sch\00f6nberg\\CD 01\\07 - 3 Pieces for Chamber Orchestra - II. M\00e1ige Viertel.mp3" -> "/home/olivier/var/Wine.sh/Apps/foobar2000 v1.4.3.wine/dosdevices/z:/home/olivier/Musique/Sch\xc3\xb6nberg, Arnold/Boulez conducts Sch\xc3\xb6nberg/CD 01/07 - 3 Pieces for Chamber Orchestra - II. M\xc3\xa1ige Viertel.mp3" (...) --- end output --- Notice the difference in non-ASCII characters encoding of the filename passed to the functions (Sch\00c3\00b6nberg versus Sch\00f6nberg). U+00f6 is Unicode code point and c3 b6 the utf-8 hexadecimal representation of 'ö'. This discussion on archlinux forums seems somewhat related: https://bbs.archlinux.org/viewtopic.php?id=149043 What is the content of your /etc/fstab (for the partition where the audio files are stored)? -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=46755 Daniil Zhilin <just4steam778(a)gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #64239|0 |1 is obsolete| | --- Comment #10 from Daniil Zhilin <just4steam778(a)gmail.com> --- Comment on attachment 64239 --> https://bugs.winehq.org/attachment.cgi?id=64239 WINEDEBUG=+file output Whoops, I'm pretty sure I was just using a locale I hadn't generated. Everything works fine now, thanks a lot for the help! -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=46755 debilerpc(a)web.de changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |NOTOURBUG Status|UNCONFIRMED |RESOLVED --- Comment #11 from debilerpc(a)web.de --- Hi, I figured out the reason for this problem on my system, which had nothing to do with Wine, although Wine was the only place where it manifested. During an Arch Linux a system update, it seems the deprecated file ~/.config/plasma-locale-settings.sh became converted to ~/.config/plasma-localerc, and thus became activated again. At least that's my theory why the problem only manifested after the last update, and why both files were present. More info: https://wiki.archlinux.org/index.php/locale#My_system_is_still_using_wrong_l... This file contained the invalid locale en_DE for unclear reasons. What it did was setting the user locale to en_DE after logging into Plasma Desktop. It seems every app *except* Wine defaulted to a valid locale. Wine however defaulted to ASCII. The problem completely went away after deleting ~/.config/plasma-localerc. -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
https://bugs.winehq.org/show_bug.cgi?id=46755 --- Comment #12 from Olivier F. R. Dierick <o.dierick(a)piezo-forte.be> --- Hello, (In reply to Daniil Zhilin from comment #10)
Comment on attachment 64239 [details] WINEDEBUG=+file output
Whoops, I'm pretty sure I was just using a locale I hadn't generated. Everything works fine now, thanks a lot for the help!
(In reply to debilerpc from comment #11)
Hi, I figured out the reason for this problem on my system, which had nothing to do with Wine, although Wine was the only place where it manifested.
I'm glad to hear that the issue is fixed for both of you. Regards -- Do not reply to this email, post in Bugzilla using the above URL to reply. You are receiving this mail because: You are watching all bug changes.
participants (1)
-
wine-bugs＠winehq.org