I have a working implementation of the new xaudio2 API in my tree. Microsoft is treating it as a replacement for dsound[1], and as a result, this new API is used by lots of recent games; see Bug 26808[2] for some examples.
Most games that use the xaudio2 API use a version of Microsoft's WMA codec. Wine doesn't currently have the ability to decode this audio for playback. In order for the xaudio2 implementation to be useful for most games, we'll need some way to convert it. I chose to use the FFmpeg library to decode WMA audio.
This means for most games, Wine's built-in xaudio2 will require that the 32-bit FFmpeg library be available on the system to successfully play audio.
The new code only uses the audio decoding features of FFmpeg, specifically linking only against libavcodec and libavutil. The newest APIs that we use were introduced in 2013. Most of the rest date from 2011 and a few from 2002. FFmpeg's API isn't stable, but I don't expect much maintenance burden from the small piece that we're using.
An additional complication is the libav fork. The APIs that we're using are currently unchanged between FFmpeg and libav, so libav should function as a drop-in replacement. Further, distros seem to be migrating back to FFmpeg, if they ever changed. So again, I don't expect this to be a barrier.
Wine packagers and developers, do you have any thoughts or objections to depending on FFmpeg/libav's libavutils and libavcodec for xaudio2 support?
[1] https://msdn.microsoft.com/en-us/library/windows/desktop/ee415813%28v=vs.85%...
[2] https://bugs.winehq.org/show_bug.cgi?id=26808
Thanks, Andrew
On 8/20/15 2:47 PM, Andrew Eikum wrote:
Wine packagers and developers, do you have any thoughts or objections to depending on FFmpeg/libav's libavutils and libavcodec for xaudio2 support?
The FFmpeg web-site says FFmpeg may be found in both GPL and LGPL forms:
" FFmpeg is licensed under the GNU Lesser General Public License (LGPL) version 2.1 http://www.gnu.org/licenses/old-licenses/lgpl-2.1.html or later. However, FFmpeg incorporates several optional parts and optimizations that are covered by the GNU General Public License (GPL) version 2 http://www.gnu.org/licenses/old-licenses/gpl-2.0.html or later. If those parts get used the GPL applies to all of FFmpeg."
It seems difficult to say when wine loads it which license applies. I think this is an area where things may not be entirely clear license-wise, but couldn't that pose a problem?
On Thu, Aug 20, 2015 at 03:10:17PM -0500, Josh DuBois wrote:
On 8/20/15 2:47 PM, Andrew Eikum wrote:
Wine packagers and developers, do you have any thoughts or objections to depending on FFmpeg/libav's libavutils and libavcodec for xaudio2 support?
The FFmpeg web-site says FFmpeg may be found in both GPL and LGPL forms:
" FFmpeg is licensed under the GNU Lesser General Public License (LGPL) version 2.1 http://www.gnu.org/licenses/old-licenses/lgpl-2.1.html or later. However, FFmpeg incorporates several optional parts and optimizations that are covered by the GNU General Public License (GPL) version 2 http://www.gnu.org/licenses/old-licenses/gpl-2.0.html or later. If those parts get used the GPL applies to all of FFmpeg."
It seems difficult to say when wine loads it which license applies. I think this is an area where things may not be entirely clear license-wise, but couldn't that pose a problem?
Thanks for looking into this. Looks like the license of the resulting binary is configurable at build-time. On Arch Linux, at least, FFmpeg is built with GPL3 code, so Wine can't use the binaries. Arguably, packagers would have to be aware of this and provide two FFmpegs, one under each license. I don't think anyone's going to like that.
This might be another argument in favor of pulling the WMA code out, as the WMA code is LGPL.
Andrew
On 08/21/2015 02:41 PM, Andrew Eikum wrote:
On Thu, Aug 20, 2015 at 03:10:17PM -0500, Josh DuBois wrote:
On 8/20/15 2:47 PM, Andrew Eikum wrote:
Wine packagers and developers, do you have any thoughts or objections to depending on FFmpeg/libav's libavutils and libavcodec for xaudio2 support?
The FFmpeg web-site says FFmpeg may be found in both GPL and LGPL forms:
" FFmpeg is licensed under the GNU Lesser General Public License (LGPL) version 2.1 http://www.gnu.org/licenses/old-licenses/lgpl-2.1.html or later. However, FFmpeg incorporates several optional parts and optimizations that are covered by the GNU General Public License (GPL) version 2 http://www.gnu.org/licenses/old-licenses/gpl-2.0.html or later. If those parts get used the GPL applies to all of FFmpeg."
It seems difficult to say when wine loads it which license applies. I think this is an area where things may not be entirely clear license-wise, but couldn't that pose a problem?
Thanks for looking into this. Looks like the license of the resulting binary is configurable at build-time. On Arch Linux, at least, FFmpeg is built with GPL3 code, so Wine can't use the binaries. Arguably, packagers would have to be aware of this and provide two FFmpegs, one under each license. I don't think anyone's going to like that.
Why would that be an issue for the Wine project? The Wine binaries as shipped by Arch Linux (basically a Wine "fork") would be coerced to a GPL3 license. But that wouldn't transfer back to the Wine project.
This might be another argument in favor of pulling the WMA code out, as the WMA code is LGPL.
bye michael
On 08/20/2015 02:47 PM, Andrew Eikum wrote:
Wine packagers and developers, do you have any thoughts or objections to depending on FFmpeg/libav's libavutils and libavcodec for xaudio2 support?
ffmpeg is not free (re: patents). Could you rework it to use gstreamer 1.x instead?
On 08/20/2015 10:18 PM, Michael Cronenworth wrote:
On 08/20/2015 02:47 PM, Andrew Eikum wrote:
Wine packagers and developers, do you have any thoughts or objections to depending on FFmpeg/libav's libavutils and libavcodec for xaudio2 support?
ffmpeg is not free (re: patents). Could you rework it to use gstreamer 1.x instead?
gstreamer is unusable for Wine due to it relying on glib and that is broken since glib-2.32.0: https://bugs.winehq.org/show_bug.cgi?id=30557
The way forward is to replace winegstreamer too.
bye michael
On Fri, 21 Aug 2015, Michael Stefaniuc wrote: [...]
gstreamer is unusable for Wine due to it relying on glib and that is broken since glib-2.32.0: https://bugs.winehq.org/show_bug.cgi?id=30557
The way forward is to replace winegstreamer too.
I think a better way forward would be to fix the issue with GLib (either from the GLib side or the Wine one) and to upgrade winegstreamer to a non-obsolete version of GStreamer.
On Thu, Aug 20, 2015 at 03:18:46PM -0500, Michael Cronenworth wrote:
On 08/20/2015 02:47 PM, Andrew Eikum wrote:
Wine packagers and developers, do you have any thoughts or objections to depending on FFmpeg/libav's libavutils and libavcodec for xaudio2 support?
ffmpeg is not free (re: patents). Could you rework it to use gstreamer 1.x instead?
In addition to what Michael Stefaniuc said, I don't think gstreamer has WMA support?
http://gstreamer.freedesktop.org/documentation/plugins.html
I wasn't able to find any open source libraries to decode WMA other than FFmpeg.
Andrew
On 08/21/2015 08:59 AM, Andrew Eikum wrote:
In addition to what Michael Stefaniuc said, I don't think gstreamer has WMA support?
http://gstreamer.freedesktop.org/documentation/plugins.html
I wasn't able to find any open source libraries to decode WMA other than FFmpeg.
Andrew
WMA support is provided by the libav plugin.
$ gst-inspect-1.0 | grep -i wma typefindfunctions: video/x-ms-asf: asf, wm, wma, wmv libav: avenc_wmav1: libav Windows Media Audio 1 encoder libav: avenc_wmav2: libav Windows Media Audio 2 encoder libav: avdec_wmalossless: libav Windows Media Audio Lossless decoder libav: avdec_wmapro: libav Windows Media Audio 9 Professional decoder libav: avdec_wmav1: libav Windows Media Audio 1 decoder libav: avdec_wmav2: libav Windows Media Audio 2 decoder libav: avdec_wmavoice: libav Windows Media Audio Voice decoder
If you go down the route of linking against ffmpeg or including a WMA decoder those of us that package wine in distributions released in patent-friendly countries will have to spend time striping out that support (if it is not a configure time option).
On Fri, Aug 21, 2015 at 09:09:31AM -0500, Michael Cronenworth wrote:
On 08/21/2015 08:59 AM, Andrew Eikum wrote:
In addition to what Michael Stefaniuc said, I don't think gstreamer has WMA support?
http://gstreamer.freedesktop.org/documentation/plugins.html
I wasn't able to find any open source libraries to decode WMA other than FFmpeg.
Andrew
WMA support is provided by the libav plugin.
$ gst-inspect-1.0 | grep -i wma typefindfunctions: video/x-ms-asf: asf, wm, wma, wmv libav: avenc_wmav1: libav Windows Media Audio 1 encoder libav: avenc_wmav2: libav Windows Media Audio 2 encoder libav: avdec_wmalossless: libav Windows Media Audio Lossless decoder libav: avdec_wmapro: libav Windows Media Audio 9 Professional decoder libav: avdec_wmav1: libav Windows Media Audio 1 decoder libav: avdec_wmav2: libav Windows Media Audio 2 decoder libav: avdec_wmavoice: libav Windows Media Audio Voice decoder
If you go down the route of linking against ffmpeg or including a WMA decoder those of us that package wine in distributions released in patent-friendly countries will have to spend time striping out that support (if it is not a configure time option).
Did libav drop whatever patent-encumbered code from ffmpeg? If not, I don't see how this solves any patent issues for distros that care about it.
Andrew
On 08/21/2015 09:18 AM, Andrew Eikum wrote:
Did libav drop whatever patent-encumbered code from ffmpeg? If not, I don't see how this solves any patent issues for distros that care about it.
GStreamer plugins can be shipped outside of the distribution.
Example: Fedora ships gstreamer with patent free codecs. RPMFusion ships additional patent encumbered plugins that Fedora users can install.
* Install Fedora * dnf install gstreamer1-plugins-good * Install RPMFusion repository * dnf install gstreamer1-libav
On Fri, Aug 21, 2015 at 09:25:46AM -0500, Michael Cronenworth wrote:
On 08/21/2015 09:18 AM, Andrew Eikum wrote:
Did libav drop whatever patent-encumbered code from ffmpeg? If not, I don't see how this solves any patent issues for distros that care about it.
GStreamer plugins can be shipped outside of the distribution.
Example: Fedora ships gstreamer with patent free codecs. RPMFusion ships additional patent encumbered plugins that Fedora users can install.
- Install Fedora
- dnf install gstreamer1-plugins-good
- Install RPMFusion repository
- dnf install gstreamer1-libav
OK. So most useful to you would be an external WMA decoding implementation and a run-time switch to enable WMA decoding iff the library is available. That way users can have WMA support by just installing a non-free package, without having to rebuild wine with a different configure switch. I'll keep that in mind, thanks.
Andrew
FFmpeg doesn't have a particularly stable API or ABI. Dynamically linking to it isn't that great of an idea because the .so number version increments fairly often requiring at least a relink. The audio decode routine is up to avcodec_decode_audio4 by now, and who knows when avcodec_decode_audio5 will happen. The deprecated functions do eventually get removed and distros can have them compiled out.
It's a really good lib if you can require specific versions, and either static-link it or ship with the version you build with, but I imagine the constantly-breaking ABI would be more of a headache for a codebase like Wine unless you directly copy over the code (but then, why not just take the decoder routines ffmpeg has and wrap them into whatever Windows audio decoder API?).
Another issue is that ffmpeg requires C99 to build, whereas Wine doesn't.
On Fri, Aug 21, 2015 at 01:32:35AM -0700, Chris Robinson wrote:
FFmpeg doesn't have a particularly stable API or ABI. Dynamically linking to it isn't that great of an idea because the .so number version increments fairly often requiring at least a relink. The audio decode routine is up to avcodec_decode_audio4 by now, and who knows when avcodec_decode_audio5 will happen. The deprecated functions do eventually get removed and distros can have them compiled out.
It's a really good lib if you can require specific versions, and either static-link it or ship with the version you build with, but I imagine the constantly-breaking ABI would be more of a headache for a codebase like Wine unless you directly copy over the code (but then, why not just take the decoder routines ffmpeg has and wrap them into whatever Windows audio decoder API?).
Hmm, okay, thanks. We really only need the WMA decoder. That probably wouldn't be too hard to pull into Wine, but I was hoping to avoid it for all the obvious reasons. If it's our best option, though, then maybe that's what we should do.
Andrew
Hello,
Most Linux distributions using libav have voted to move to ffmpeg in the future, Debian (and thus Ubuntu) being one of the most prominent ones.
Even with ffmpeg API changes, remaining source-compatible should not be a problem for wine. But I guess the problem is not source-compatibility for wine but binary-compatibility for CrossOver. In that case, I'd suggest dlopen'ing and checking for the symbol you want. Maybe even have different code paths if avcodec_decode_audio5 is available (that means a later version of ffmpeg is available on that system).
Or you could forget about everything and just take libwmapro from RockBox:
https://github.com/mguentner/rockbox/tree/master/apps/codecs/libwmapro
They took ffmpeg, isolated the WMA code and made it a standalone library. They even further optimized the code. It's taken from an old version of ffmpeg (2010) but it works :-)
On Thu, Aug 20, 2015 at 9:47 PM, Andrew Eikum aeikum@codeweavers.com wrote:
I have a working implementation of the new xaudio2 API in my tree. Microsoft is treating it as a replacement for dsound[1], and as a result, this new API is used by lots of recent games; see Bug 26808[2] for some examples.
Most games that use the xaudio2 API use a version of Microsoft's WMA codec. Wine doesn't currently have the ability to decode this audio for playback. In order for the xaudio2 implementation to be useful for most games, we'll need some way to convert it. I chose to use the FFmpeg library to decode WMA audio.
This means for most games, Wine's built-in xaudio2 will require that the 32-bit FFmpeg library be available on the system to successfully play audio.
The new code only uses the audio decoding features of FFmpeg, specifically linking only against libavcodec and libavutil. The newest APIs that we use were introduced in 2013. Most of the rest date from 2011 and a few from 2002. FFmpeg's API isn't stable, but I don't expect much maintenance burden from the small piece that we're using.
An additional complication is the libav fork. The APIs that we're using are currently unchanged between FFmpeg and libav, so libav should function as a drop-in replacement. Further, distros seem to be migrating back to FFmpeg, if they ever changed. So again, I don't expect this to be a barrier.
Wine packagers and developers, do you have any thoughts or objections to depending on FFmpeg/libav's libavutils and libavcodec for xaudio2 support?
[1] https://msdn.microsoft.com/en-us/library/windows/desktop/ee415813%28v=vs.85%...
[2] https://bugs.winehq.org/show_bug.cgi?id=26808
Thanks, Andrew
On 08/22/2015 04:27 PM, Pau Garcia i Quiles wrote:
But I guess the problem is not source-compatibility for wine but binary-compatibility for CrossOver. In that case, I'd suggest dlopen'ing and checking for the symbol you want.
There's also issues with structure changes which won't be caught that way. libwmapro sounds like the sanest solution, short of doing the same thing with Wine.
On Sun, Aug 23, 2015 at 03:44:13AM -0700, Chris Robinson wrote:
On 08/22/2015 04:27 PM, Pau Garcia i Quiles wrote:
But I guess the problem is not source-compatibility for wine but binary-compatibility for CrossOver. In that case, I'd suggest dlopen'ing and checking for the symbol you want.
There's also issues with structure changes which won't be caught that way. libwmapro sounds like the sanest solution, short of doing the same thing with Wine.
Cool, I didn't know about libwmapro. It does look like exactly what we need, but I'm concerned about it being so out of date. ffmpeg's wma files alone have over 400 commits since 2011.
Andrew
On Mon, Aug 24, 2015 at 4:09 PM, Andrew Eikum aeikum@codeweavers.com wrote:
Cool, I didn't know about libwmapro. It does look like exactly what we
need, but I'm concerned about it being so out of date. ffmpeg's wma files alone have over 400 commits since 2011.
I don't know what was added later in ffmpeg but libwmapro definitely plays WMA files on my Sansa Clip+ running RockBox 3.13.
This is what I would do:
- Take libwmapro as-is to replace the ffmpeg dependency in your xinput2 implementation
- If something is missing, or does not work as expected, contact the libwmapro maintainers and ask them why they haven't kept up to date with ffmpeg. Maybe those changes are only adaptations to new ffmpeg APIs, or unimportant changes. And maybe they will even say "it's a good idea, we'll look into updating libwmapro"
- If everything else fails, take ffmpeg and spend a non-negligible amount of time isolating a "libwinewma", which you'll have to maintain yourself and everything in the future.
-
Andrew Eikum -
Chris Robinson -
Francois Gouget -
Josh DuBois -
Michael Cronenworth -
Michael Stefaniuc -
Pau Garcia i Quiles