Major mmdevapi and winmm audio bugs
Hi, instead of Wine Weekly News (WWN, any taker?) you get monthly audio news. The surprising thing is: it looks like most audio bugs remaining in bugzilla are from the pre-mmdevapi era and have not been updated since mid-2011. - Using devices other than "default" (dmix or pulse): #29294 no sound with ALSA loopback -- are they duplicates of each other #28781 loopback to Jack -- or is one about DSound only? We have are several issues about crackling sound: - One definitely is caused by CreateTimerQueue which is not regular enough or does not deliver 10ms, but 12ms callbacks. This is in #28723 which includes a work-around in comment #130 That may affect the Solaris machine on test.winehq.org too. - One is caused by a mmdevapi design error: the audio buffer we use is simply too small, and Linux can't bear a regular 10ms rate. Things got worse since the introduction of the Completely Fair Scheduler. It can't even guarantee a regular 50ms one. There's a need for a much larger buffer. That's why wine < 1.3.25 is giving steadier sound. #29531 may be caused by this. - Mac OS X 10.5.8 crackles constantly with dlls/mmdevapi/tests/render I'd like somebody with 10.6 or .7 to perform the interactive test. rm -f render.ok; WINETEST_INTERACTIVE=1 make render.ok - Lastly, there's PulseAudio #28856 among others I don't understand why I couldn't find bugs in PA's bugzilla, so I wrote some: - https://bugs.freedesktop.org/show_bug.cgi?id=46296 About the need to restart the PA server from time to time - https://bugs.freedesktop.org/show_bug.cgi?id=46297 PA's huge buffering causes issues with winmm - https://bugs.freedesktop.org/show_bug.cgi?id=46412 How to obtain PA's typical 44100 rate? Feel free to contribute if you know apps outside Wine with similar issues. Surprisingly, users have reported improvements in the latest wine-1.4rcX even when nothing changed, but also mentioned updates on the PA and alsa_plugins side. Perhaps that improved the situation. I was pleased to see Wine's testsuite help find bugs in Pulse: http://lists.freedesktop.org/archives/pulseaudio-discuss/2012-February/01282... There are issues not always associated with a bug entry: - #29472 comment #12 DSound GetDevicePosition is not how it should be - Check OSS4 with and without vmix #28056 #28790 - winmm handling huge latency (such as PA's) & when to send notifications - Issues with microphone capture? - DSound mixing and buffer sizes & padding vs. GetPosition - Deal with intermittent errors from ALSA etc. in mmdevapi, winmm and dsound - Device enumeration and PA preventing access to the underlying device Is this known to cause no sound in an app (or only in our wave tests?) - various missing trailing sound with DSound #21209 - #29369 strange issue involving PA and perhaps messaging - misc. losing sound: #28319 #25164 #28770 - lead-out needed with dmix - Use lock-less design to reduce contention, cf. #29531, comment #7 - multi-channel, 5:1 etc. in mmdevapi and DSound - Bugs in the quartz or winegstreamer area that I don't have on my radar Of course there are more bugs than this. Regards, Jörg Höhle
On Fri, Feb 24, 2012 at 13:28, <Joerg-Cyril.Hoehle(a)t-systems.com> wrote:
This one is a question, not a bug, so should rather be asked on PA official support channels TBH... Frédéric
On Friday, February 24, 2012 1:28:48 PM Joerg-Cyril.Hoehle(a)t-systems.com wrote:
- https://bugs.freedesktop.org/show_bug.cgi?id=46296 About the need to restart the PA server from time to time - https://bugs.freedesktop.org/show_bug.cgi?id=46297 PA's huge buffering causes issues with winmm
I think these two have the same cause. The minimum latency (the minreq or period size, essentially) PulseAudio will allow steadily grows while the Pulse server is used, necessitating its restart to get it back down to manageable levels. ALSA's pulse plugin seems to lie and report that it gives you the buffer metrics you asked for, but has a larger buffer behind the scenes to deal with the server's out-of-control minimum latency requirement.
Hey Joerg, Op 24-02-12 13:28, Joerg-Cyril.Hoehle(a)t-systems.com schreef:
Hi,
instead of Wine Weekly News (WWN, any taker?) you get monthly audio news.
The surprising thing is: it looks like most audio bugs remaining in bugzilla are from the pre-mmdevapi era and have not been updated since mid-2011.
- Using devices other than "default" (dmix or pulse): #29294 no sound with ALSA loopback -- are they duplicates of each other #28781 loopback to Jack -- or is one about DSound only?
We have are several issues about crackling sound:
- One definitely is caused by CreateTimerQueue which is not regular enough or does not deliver 10ms, but 12ms callbacks. This is in #28723 which includes a work-around in comment #130 That may affect the Solaris machine on test.winehq.org too.
- One is caused by a mmdevapi design error: the audio buffer we use is simply too small, and Linux can't bear a regular 10ms rate. Things got worse since the introduction of the Completely Fair Scheduler. It can't even guarantee a regular 50ms one. There's a need for a much larger buffer. That's why wine < 1.3.25 is giving steadier sound. #29531 may be caused by this. Really? I would say using rtkit for setting threads to realtime would be an aid for this.. - Mac OS X 10.5.8 crackles constantly with dlls/mmdevapi/tests/render I'd like somebody with 10.6 or .7 to perform the interactive test. rm -f render.ok; WINETEST_INTERACTIVE=1 make render.ok
- Lastly, there's PulseAudio #28856 among others I don't understand why I couldn't find bugs in PA's bugzilla, so I wrote some: - https://bugs.freedesktop.org/show_bug.cgi?id=46296 About the need to restart the PA server from time to time - https://bugs.freedesktop.org/show_bug.cgi?id=46297 PA's huge buffering causes issues with winmm - https://bugs.freedesktop.org/show_bug.cgi?id=46412 How to obtain PA's typical 44100 rate? Feel free to contribute if you know apps outside Wine with similar issues. Surprisingly, users have reported improvements in the latest wine-1.4rcX even when nothing changed, but also mentioned updates on the PA and alsa_plugins side. Perhaps that improved the situation.
I was pleased to see Wine's testsuite help find bugs in Pulse: http://lists.freedesktop.org/archives/pulseaudio-discuss/2012-February/01282... bug 46412 is worded wrong, pulseaudio can get native format just fine with PA_STREAM_FIX_RATE|PA_STREAM_FIX_FORMAT|PA_STREAM_FIX_CHANNELS, but maybe the pulse plugin doesn't respect it.
fwiw, IAudioClient::GetMixFormat works just fine with my winepulse driver, dsound doesn't seem to respect the rate, however. If you feel like toying around, my git tree is at: http://repo.or.cz/w/wine/multimedia.git and contains a simple winepulse driver that sadly doesn't pass all tests yet, but I've not been motivated to fix the remaining ones since nobody seems to care about the driver and it will never get merged. Most test failures are related to IAudioClock including silence time. Hacked away a bit more and I think I can fix it without being required to have a shadow buffer, using the write callback to signal when writing is done and keeping more track. The current "le sigh" commit solves it in a wrong way, but it's just a first stab at solving it. I don't trust lockless, and I'm just using a simple mainloop thread based that matches the one pulseaudio uses everywhere, but manually created with CreateThread so normal wine calls would still work from it without weird bugs popping up.
There are issues not always associated with a bug entry: - #29472 comment #12 DSound GetDevicePosition is not how it should be - Check OSS4 with and without vmix #28056 #28790 - winmm handling huge latency (such as PA's) & when to send notifications - Issues with microphone capture? - DSound mixing and buffer sizes & padding vs. GetPosition - Deal with intermittent errors from ALSA etc. in mmdevapi, winmm and dsound - Device enumeration and PA preventing access to the underlying device Is this known to cause no sound in an app (or only in our wave tests?) - various missing trailing sound with DSound #21209 - #29369 strange issue involving PA and perhaps messaging - misc. losing sound: #28319 #25164 #28770 - lead-out needed with dmix - Use lock-less design to reduce contention, cf. #29531, comment #7 - multi-channel, 5:1 etc. in mmdevapi and DSound - Bugs in the quartz or winegstreamer area that I don't have on my radar
Winepulse supports 5.1 just fine, but dsound does not. I'm just using the dsound-openal replacement for now in games. It's a slightly patched version of http://repo.or.cz/w/dsound-openal.git ~Maarten
Hi, Maarten Lankhorst wrote (partly in a former thread):
R. thread priority -- no "Pro Audio" / Real-Time priority Not going to happen, ever. AJ nuked all my attempts at it, [...] Or just fix things properly and use rtkit for time critical threads and now: I would say using rtkit for setting threads to realtime would be an aid for this.
I can understand AJ's reluctance to RT code in Wine. I wouldn't feel comfortable either. I believe few people realize that RT is a gifted gift. Bear in mind that RT processes work at a priority above any root or user process. Do something wrong there, consume 100% CPU and you can press the single core machine's power down button. I would not accept RT code in a system without a very careful code review and think twice about the consequences, and require a design that keeps the RT part of the app as small as possible, better separated. OTOH, RT is the means to provide reliable ticks, and that's why PA as a sound server wants that. Now one can argue that Wine's mmdevapi plays sound server for all w32 apps, so it has the same needs.
I'm just using a simple mainloop thread That's my plan for past 1.4 too. Get rid of CreateTimerQueue as callback source in mmdevapi, use a thread for the feeder and poll() to wait 10ms, like all devices used to do. I believe that's too big a change for code freeze.
W.r.t. to 1.4.0, I believe the following path should be taken: - Write CreateTimerQueue tests to verify whether native behaves like: a) sleep(Xms) /* what Wine does now */ or b) sleep(Xms - elapsed) /* what winmm:timeSetEvent does */ - If b), fix CreateTimerQueue. - If a), CreateTimerQueue is unusable for the purposes of Wine's mmdevapi and something else must be sought... Regards, Jörg Höhle
Hey Joerg, Op 27-02-12 19:22, Joerg-Cyril.Hoehle(a)t-systems.com schreef:
Hi,
Maarten Lankhorst wrote (partly in a former thread):
R. thread priority -- no "Pro Audio" / Real-Time priority Not going to happen, ever. AJ nuked all my attempts at it, [...] Or just fix things properly and use rtkit for time critical threads and now: I would say using rtkit for setting threads to realtime would be an aid for this. I can understand AJ's reluctance to RT code in Wine. I wouldn't feel comfortable either. I believe few people realize that RT is a gifted gift. Bear in mind that RT processes work at a priority above any root or user process. Do something wrong there, consume 100% CPU and you can press the single core machine's power down button. I would not accept RT code in a system without a very careful code review and think twice about the consequences, and require a design that keeps the RT part of the app as small as possible, better separated.
OTOH, RT is the means to provide reliable ticks, and that's why PA as a sound server wants that. Now one can argue that Wine's mmdevapi plays sound server for all w32 apps, so it has the same needs. Agreed, I've tried to keep it as small as possible for the thread itself, the mainloop thread calls very few functions, I need to add a bit more, but the only source of problems could come from the pulseaudio lib itself.
At this point my winepulse driver is very reliable for rendering, and capture should be the same once I add the queue stuff, it would also be the only driver to actually pass the tests. For capture I plan to just add 2 queues, one for readable packets, and one for packets that can be filled for reading. With that approach I should be able to pass all tests in the most efficient way while keeping the metadata mmdevapi adds for capture.
I'm just using a simple mainloop thread That's my plan for past 1.4 too. Get rid of CreateTimerQueue as callback source in mmdevapi, use a thread for the feeder and poll() to wait 10ms, like all devices used to do. I believe that's too big a change for code freeze.
W.r.t. to 1.4.0, I believe the following path should be taken: - Write CreateTimerQueue tests to verify whether native behaves like: a) sleep(Xms) /* what Wine does now */ or b) sleep(Xms - elapsed) /* what winmm:timeSetEvent does */ - If b), fix CreateTimerQueue. - If a), CreateTimerQueue is unusable for the purposes of Wine's mmdevapi and something else must be sought... Even if so, it is unneeded for the pulse driver, I'm using the auto timing update feature and the callback for when rendering is not running. When the buffer is not underflowed, I'm using the write callback to signal more data is available.
Timerqueue would be a lie, the driver should schedule when data can actually be written and not based purely on ticks. Pulseaudio allows this and I'm using this feature in my driver. It passes all the render tests now except the volume related ones (they're a reminder, and a nop on other drivers anyhow), and I'm working to complete capture in a similar fashion as well. I believe the event should be triggered when there is actual data available, at least that's how I understood the event is supposed to work. Doing it based on a timer that isn't tied to the state in any way feels wrong. Having 2 threads for pulse feels wrong too, especially since it can be done with just using the one pulse provides, because you can set callbacks for when more data can be read or written. ~Maarten
On Mon, Feb 27, 2012 at 11:41:56PM +0100, Maarten Lankhorst wrote:
W.r.t. to 1.4.0, I believe the following path should be taken: - Write CreateTimerQueue tests to verify whether native behaves like: a) sleep(Xms) /* what Wine does now */ or b) sleep(Xms - elapsed) /* what winmm:timeSetEvent does */ - If b), fix CreateTimerQueue. - If a), CreateTimerQueue is unusable for the purposes of Wine's mmdevapi and something else must be sought... Even if so, it is unneeded for the pulse driver, I'm using the auto timing update feature and the callback for when rendering is not running. When the buffer is not underflowed, I'm using the write callback to signal more data is available.
Timerqueue would be a lie, the driver should schedule when data can actually be written and not based purely on ticks. Pulseaudio allows this and I'm using this feature in my driver. It passes all the render tests now except the volume related ones (they're a reminder, and a nop on other drivers anyhow), and I'm working to complete capture in a similar fashion as well.
I believe the event should be triggered when there is actual data available, at least that's how I understood the event is supposed to work. Doing it based on a timer that isn't tied to the state in any way feels wrong. Having 2 threads for pulse feels wrong too, especially since it can be done with just using the one pulse provides, because you can set callbacks for when more data can be read or written.
Unfortunately, the only backend that actually produces that kind of information is Pulse. So we need a solution for the other drivers anyway. At that point, we'll have to decide whether we want to use the same mechanism for Pulse, or rely on the callbacks. There's an argument for having each driver behave as similar as possible, even if it's arguably less "correct." I've also been burned enough times by relying on the audio backend to work correctly that I hesitate to give the backends more responsibility than absolutely necessary. I'm investigating native TimerQueue's operation now. If it turns out that TimerQueue isn't sufficient, we'll probably just switch over to using poll() like winmm's timer stuff does. Andrew
On Tue, Feb 28, 2012 at 08:24:37AM -0600, Andrew Eikum wrote:
I'm investigating native TimerQueue's operation now. If it turns out that TimerQueue isn't sufficient, we'll probably just switch over to using poll() like winmm's timer stuff does.
And, not too surprisingly, TimerQueue isn't sufficient. I've attached my tests here. The tests ask to execute the callback every 12 ms. On my Windows 7 VM, I found that it executed with intervals like: 20, 10, 10, 10, 10, 20, 10, 10, 10, 10, ... With an 11 occasionally interspersed. So, forget that. Winmm's timer functions use poll() with a timeout value, subtracting the time elapsed curing the callback. This works quite well in dsound. The Win32 API also provides the SetTimer API. But that depends on a message queue, which is a hassle I don't know if we want to bother with in the drivers (plus, it may be no more reliable than the TimerQueues). Since we're already using poll() in winmm and it seems to work well, that would be my suggestion. Any thoughts? Andrew
Hi Andrew, one small contribution:
Winmm's timer functions use poll() with a timeout value, subtracting the time elapsed curing the callback. This works quite well in dsound.
The Win32 API also provides the SetTimer API. But that depends on a message queue, which is a hassle I don't know if we want to bother with in the drivers (plus, it may be no more reliable than the TimerQueues).
Using SetTimer is not the way you want to go. Its timing is unpredictable and its interface strange. I'd stay away from it. --Juan
Andrew Eikum wrote:
And, not too surprisingly, TimerQueue isn't sufficient. I've attached my tests here. The tests ask to execute the callback every 12 ms. On my Windows 7 VM, I found that it executed with intervals like: 20, 10, 10, 10, 10, 20, 10, 10, 10, 10, ... With an 11 occasionally interspersed. So, forget that.
No. Did you notice that on *average*, a callback occurs every 12ms (5 within 60ms)? This is the same essential property that winmm:setTimer exhibits too. Wine's CreateTimerQueue is buggy, because it doesn't do that. Wine's winmm:timer does. A patch is needed for CTQ. Now if Wine's CTQ would be fixed, Wine's audio would have fewer underruns. The periodicity is another topic.
my Windows 7 VM I don't trust timings from a VM. 10/20ms may be an artefact. OTOH it may be a design decision from MS related to scheduler quantums or whatever.
Since we're already using poll() in winmm and it seems to work well, that would be my suggestion. Sure, but then you need an own thread in mmdevapi, don't you? So we're back at the point where I said earlier that I can well imagine a fix to CTQ for wine-1.4.0, but less so a move to an own thread + poll.
Regards, Jörg Höhle
Hi, Maarten Lankhorst wrote:
For capture I plan to just add 2 queues, one for readable packets, and one for packets that can be filled for reading. Winecoreaudio uses exactly this.
... for the pulse driver, I'm using the auto timing update feature and the callback for when rendering is not running. When the buffer is not underflowed, I'm using the write callback to signal more data is available. I'm not familiar with PA's API, so I'm not following you. I'm not even sure we talk about the same thing.
I believe the event should be triggered when there is actual data available, at least that's how I understood the event is supposed to work.
I distinguish 2 events: a) ALSA/PA backend ready for more data b) mmdevapi SetEventHandle to the app Note that b) only exists when using EVENTCALLBACK. We have tests about b) that show that mmdevapi delivers events to the app even when stopped (Wine fails those with todo_wine). I don't know at what rate. Regarding a), nothing prevents you from using the back-end's favourite notification mechanism to wake up - or use a timer. I agree with you that ideally, somehow, b) and a) should be in sync, otherwise we'll have clock skew and the XAudio scenario will exhibit occasional underruns. But there are big HOWEVER: - ALSA/PA may not grant us the period we'd like (10ms). 21.333ms is common with ALSA/dmix. I don't know how important the 10ms are. You're familiar with DSound, so perhaps you have an opinion why DSound has been using 10ms intervals in Wine regardless of the back-end's period. I believe that Wine ought to use the same rate as native or some app will exhibit a bug. - We do not need 10ms events when not using EVENTCALLBACK, so why not write all available data (e.g. 333ms from winmm:PlaySound) and sleep that much? Laptop owners will thank us. - How to continue signaling type a events after IAudioClient_Stop? By continuing to write silence to the back-end?!? - Last but not least, I don't know whether ALSA's signaling and mmdevapi's SetEvent needs can be made compatible even when the periods would agree. mmdevapi's SetEvent means "don't worry, you have one period time to write data" or even "you may write data, but I already have buffered some". ALSA/PA's signal means(?) "hurry up, you're on the verge of underrun". Please correct me. - The test_worst_case aka. XAudio scenario has tough requirements: - Signal events not often enough and you'll enter an underrun. - Signal events too early and it'll write no data. The event was wasted and the next one may come too late to avoid an underrun. In summary, if the back-end guarantees 10ms wake-ups, you should be able to nicely sync a) and b) (probably by writing silence when stopped...) If not, then some mechanism must be there to decouple the two. So far we've followed the "decouple" route in winealsa/oss. I believe the key to the solution in mmdevapi will come from both being able to vary the sleeping intervals to adapt to clock skew *and* not be bound to be exactly in sync with the back-end, thanks to a latency-inducing buffer large enough to compensate for the variability. IOW, the design goal is to reach during normal playback a state where: Event received => GetCurrentPadding has (or pretends to have) decreased by at least 1 period (which implies there's room for writing). Regards, Jörg
Hey Joerg, Having actually finished the pulseaudio implementation and trying to fix some fallout. Op 29-02-12 16:28, Joerg-Cyril.Hoehle(a)t-systems.com schreef:
Hi,
Maarten Lankhorst wrote:
For capture I plan to just add 2 queues, one for readable packets, and one for packets that can be filled for reading. Winecoreaudio uses exactly this. Working now, with timestamps so all tests pass.
IAudioClock::GetPosition I have to figure out still, I was allowing it to update based on stream time passed, but dsound fails with that, so I have to determine windows' behavior some more.
... for the pulse driver, I'm using the auto timing update feature and the callback for when rendering is not running. When the buffer is not underflowed, I'm using the write callback to signal more data is available. I'm not familiar with PA's API, so I'm not following you. I'm not even sure we talk about the same thing. Because pulseaudio receives all data on mainloop you can specify callbacks that will be run when those events are received too. I'm using the latency callback when first starting to signal the event when no data is playing. When data is buffered I use the pulse_rd_callback which fires every period because of pa_stream_early_requests. pulse_wr_callback is fired when there's at least a period buffered.
I believe the event should be triggered when there is actual data available, at least that's how I understood the event is supposed to work. I distinguish 2 events: a) ALSA/PA backend ready for more data b) mmdevapi SetEventHandle to the app
Note that b) only exists when using EVENTCALLBACK.
We have tests about b) that show that mmdevapi delivers events to the app even when stopped (Wine fails those with todo_wine). I don't know at what rate. Presumably at the same rate, but since in those cases you have no data I doubt it matters if you fire less often.
Regarding a), nothing prevents you from using the back-end's favourite notification mechanism to wake up - or use a timer. Which my winepulse does. :)
I agree with you that ideally, somehow, b) and a) should be in sync, otherwise we'll have clock skew and the XAudio scenario will exhibit occasional underruns.
But there are big HOWEVER: - ALSA/PA may not grant us the period we'd like (10ms). 21.333ms is common with ALSA/dmix. I don't know how important the 10ms are. You're familiar with DSound, so perhaps you have an opinion why DSound has been using 10ms intervals in Wine regardless of the back-end's period. I believe that Wine ought to use the same rate as native or some app will exhibit a bug. If dmix doesn't support 10ms, set default period to 1024 samples and use that, no point in trying to do something it's not. Don't forget to update GetStreamLatency in that case, though.
- We do not need 10ms events when not using EVENTCALLBACK, so why not write all available data (e.g. 333ms from winmm:PlaySound) and sleep that much? Laptop owners will thank us. It's possible for winmm and dsound, but the drivers will have to keep the default period in any case, for when other mmdevapi programs rely on it.
- How to continue signaling type a events after IAudioClient_Stop? By continuing to write silence to the back-end?!? Could work, I'm abusing the auto timing update from pulseaudio, which fires periodically as well, even when playing nothing.
- Last but not least, I don't know whether ALSA's signaling and mmdevapi's SetEvent needs can be made compatible even when the periods would agree. mmdevapi's SetEvent means "don't worry, you have one period time to write data" or even "you may write data, but I already have buffered some". ALSA/PA's signal means(?) "hurry up, you're on the verge of underrun". Please correct me. As far as my understanding of mmdevapi is, it fires every period. If you have some data gogogogo, but write at least GetStreamLatency worth of samples, or you'll have an underrun. That also means mmdevapi rendering will have to be periodic or bad things occur.
- The test_worst_case aka. XAudio scenario has tough requirements: - Signal events not often enough and you'll enter an underrun. - Signal events too early and it'll write no data. The event was wasted and the next one may come too late to avoid an underrun.
In summary, if the back-end guarantees 10ms wake-ups, you should be able to nicely sync a) and b) (probably by writing silence when stopped...) If not, then some mechanism must be there to decouple the two. Or never use anything but dmix/real hardware with winealsa and just use winepulse for pulseaudio. You know the period size in that case and can force a wake up. pulseaudio's alsa wrapper might have issues because it will wait until at least one frame is filled before marking it as such, in the worst case hiding the pulseaudio native length from alsa applications.
So far we've followed the "decouple" route in winealsa/oss. I believe the key to the solution in mmdevapi will come from both being able to vary the sleeping intervals to adapt to clock skew *and* not be bound to be exactly in sync with the back-end, thanks to a latency-inducing buffer large enough to compensate for the variability. IOW, the design goal is to reach during normal playback a state where: Event received => GetCurrentPadding has (or pretends to have) decreased by at least 1 period (which implies there's room for writing). I think the key is to use accurate periods and report GetStreamLatency correctly. Use it in wine where needed for winmm and dsound.
~Maarten
participants (6)
-
Andrew Eikum -
Chris Robinson -
Frédéric Delanoy -
Joerg-Cyril.Hoehle＠t-systems.com -
Juan Lang -
Maarten Lankhorst