Hi everyone,
Previously I post this question on wine-user forum, seems nobody answered/interested, so I post it here. Hope somebody give me a hand :)
I have some questions about wine video capture andrendering.
I'm a newbie to wine, and am working on QQ or any other messengers in wine. My environment is : ubuntu 10.04, installed wine 1.1.42 (later when I modify code, wine-1.1.42's source instead), QQ2009. And during my debugging, I also check some source code from latest wine 1.2-rc4.
Although QQ's level is "garbage", but fortunately it works almost fine under wine, including login, text chat, ..... But the video call can only partially work.I can see my friend's(he's under windows of course) video, while I can't see my preview video, and he cannot see mine video too. And no audio at all. (Audio is another issue, I haven't deal with it now.)
As a comparison, I run GraphEdt to check what's wrong with video capture and rendering, my filter graph=video capture + video render. when I 'use clock', the same thing happens, no video rendering at all, but when I DONOT 'use clock', video rendering works fine.
Then I read the source code and find out one problem(maybe right, maybe wrong), when wine/dlls/qcap/v4l.c send media samples to downstream filters, it do not set any timestamp on it. Thus if I use VideoRenderer as I did in GraphEdt, the video render got wrong start and stop reference time for media sample, and then the synchronisation result will be very strange. So I modify the v4l.c and set proper timestamp on media sample. Good result, even when I 'use clock', the capture+render graph works find in GraphEdt.
But when I turn to QQ, my video capture+preview+encoding graph still seems don't work.
Then I checked msdn for these things, according to msdn: "Video frames from a preview pin (as opposed to a capture pin) are not time stamped. Because of graph latency, a video frame that is stamped with the capture time will always arrive late at the video renderer. This may cause the renderer to drop frames, in an attempt at quality control." "Quality control is a mechanism for adjusting the rate of data flow through the filter graph in response to run-time performance. If a renderer filter is receiving too much data or too little data, it can send a quality message. The quality message requests an adjustment in the data rate. By default, quality messages travel upstream from the renderer until they reach a filter that can respond (if any)."
I've cheked winedbg log, QQ's filter graph looks like: vfw capture filter + QQ's own video render filter + ... + ddraw/d3d7 rendering. QQ did not use smarttee, preview pin, standard VideoRender or VideoRenderDefault or VideoMixingRender9 (because QQ never query their id). Instead, I guess QQ write a video render filter by itself, and in QQ's render filter, it directly use ddraw/d3d7 surface to render, and maybe it does something similar to smarttee, to output to preview and encoding at the same time.
Since QQ did not use preview pin directly, at first I guess maybe 'not time stamped' is not correct, so I add some timestamp to capture's output video frame, still no preview occurs, but result in continuous IQualityControl interface query, but even latest wine-1.2-rc4 has not implemented IQualityControl, so I guess, still something wrong in timestamp synchronisation, and QQ's video render want to use IQualityControl to fix it which wine doesn't support, so maybe every frame is dropped, this result in my black preview video window.
I read from wine docs/wiki, once before windows messenger (8.1?) worked fine in wine, I guess since you have written qcap/vfwcapture, of course somebody has used wine for camera+preview+maybe encoding before, why it succeeded by then, but fail now? Is there anything wrong for me?
Btw1: there's a bug in wine-1.1.42's dlls/quartz/memallocator.c, MediaSample's GetTime() return value is wrong, I've fixed that bug according to latest wine, but still no video preview, and my friend cannot see my video.
Btw2: since qcap only works for v4l1, I ran flashcam to convert v4l2 to v4l1. But I still wondering, since Maarteen original code supports v4l2, why it result in only support v4l1 (but not both v4l1 and v4l2)?
Thank you very much in advance.
Best regards, Karma
搬迁通知： 2010年1月18日起，上海奇码数字信息有限公司迁至新址办公。 新地址为：上海张江祖冲之路887弄88号502室。 ==========================================================
Hello,
2010/6/23 shenyue shenyue@magima.com.cn:
Hi everyone,
Previously I post this question on wine-user forum, seems nobody answered/interested, so I post it here. Hope somebody give me a hand :)
I have some questions about wine video capture andrendering.
I'm a newbie to wine, and am working on QQ or any other messengers in wine. My environment is : ubuntu 10.04, installed wine 1.1.42 (later when I modify code, wine-1.1.42's source instead), QQ2009. And during my debugging, I also check some source code from latest wine 1.2-rc4.
Although QQ's level is "garbage", but fortunately it works almost fine under wine, including login, text chat, ..... But the video call can only partially work.I can see my friend's(he's under windows of course) video, while I can't see my preview video, and he cannot see mine video too. And no audio at all. (Audio is another issue, I haven't deal with it now.)
As a comparison, I run GraphEdt to check what's wrong with video capture and rendering, my filter graph=video capture + video render. when I 'use clock', the same thing happens, no video rendering at all, but when I DONOT 'use clock', video rendering works fine.
Then I read the source code and find out one problem(maybe right, maybe wrong), when wine/dlls/qcap/v4l.c send media samples to downstream filters, it do not set any timestamp on it. Thus if I use VideoRenderer as I did in GraphEdt, the video render got wrong start and stop reference time for media sample, and then the synchronisation result will be very strange. So I modify the v4l.c and set proper timestamp on media sample. Good result, even when I 'use clock', the capture+render graph works find in GraphEdt.
But when I turn to QQ, my video capture+preview+encoding graph still seems don't work.
I only made it work for msn messenger a long time ago, in 6.2 or so. I'm surprised it still works. Only change since then has been adding support for libv4l. (v4l2 with v4l1 api)
Then I checked msdn for these things, according to msdn: "Video frames from a preview pin (as opposed to a capture pin) are not time stamped. Because of graph latency, a video frame that is stamped with the capture time will always arrive late at the video renderer. This may cause the renderer to drop frames, in an attempt at quality control." "Quality control is a mechanism for adjusting the rate of data flow through the filter graph in response to run-time performance. If a renderer filter is receiving too much data or too little data, it can send a quality message. The quality message requests an adjustment in the data rate. By default, quality messages travel upstream from the renderer until they reach a filter that can respond (if any)."
I've cheked winedbg log, QQ's filter graph looks like: vfw capture filter + QQ's own video render filter + ... + ddraw/d3d7 rendering. QQ did not use smarttee, preview pin, standard VideoRender or VideoRenderDefault or VideoMixingRender9 (because QQ never query their id). Instead, I guess QQ write a video render filter by itself, and in QQ's render filter, it directly use ddraw/d3d7 surface to render, and maybe it does something similar to smarttee, to output to preview and encoding at the same time.
Could it be a ddraw/d3d7 failure rather than a wine failure?
Since QQ did not use preview pin directly, at first I guess maybe 'not time stamped' is not correct, so I add some timestamp to capture's output video frame, still no preview occurs, but result in continuous IQualityControl interface query, but even latest wine-1.2-rc4 has not implemented IQualityControl, so I guess, still something wrong in timestamp synchronisation, and QQ's video render want to use IQualityControl to fix it which wine doesn't support, so maybe every frame is dropped, this result in my black preview video window.
I read from wine docs/wiki, once before windows messenger (8.1?) worked fine in wine, I guess since you have written qcap/vfwcapture, of course somebody has used wine for camera+preview+maybe encoding before, why it succeeded by then, but fail now? Is there anything wrong for me?
Because originally I only wrote it to work with msn messenger 6.2 ;).
Btw1: there's a bug in wine-1.1.42's dlls/quartz/memallocator.c, MediaSample's GetTime() return value is wrong, I've fixed that bug according to latest wine, but still no video preview, and my friend cannot see my video.
Btw2: since qcap only works for v4l1, I ran flashcam to convert v4l2 to v4l1. But I still wondering, since Maarteen original code supports v4l2, why it result in only support v4l1 (but not both v4l1 and v4l2)?
Sort of, I didn't adjust the code for v4l2, however with libv4l it should work in linux..
~Maarten
Hi Maarten,
Thanks for reply so soon. Details reply below.
----- Original Message ----- From: "Maarten Lankhorst" m.b.lankhorst@gmail.com To: "shenyue" shenyue@magima.com.cn Cc: wine-devel@winehq.org Sent: Wednesday, June 23, 2010 10:50 PM Subject: Re: Question about video capture and rendering and encoding in wine
Hello,
I only made it work for msn messenger a long time ago, in 6.2 or so. I'm surprised it still works. Only change since then has been adding support for libv4l. (v4l2 with v4l1 api)
Could it be a ddraw/d3d7 failure rather than a wine failure?
Yes, possible. Up to now, I'm sure v4l ReadThread keeps outputing frames, but I didn't find any related Blt for video preview window, only Blt for the video of my friends. And there's so many surfaces QQ created, I can only guess which surface is for preview and remote video from the surface's DSCAPS and DS_WIDTH/HEIGHT, I didn't find any surface close to preview video's width and height.
Now since my timestamp modification finally doesn't solve QQ problem, I'm now checking whether something wrong with ddraw/d3d7.
Because originally I only wrote it to work with msn messenger 6.2 ;).
Sort of, I didn't adjust the code for v4l2, however with libv4l it should work in linux..
~Maarten
Best Regards,
Karma
搬迁通知： 2010年1月18日起，上海奇码数字信息有限公司迁至新址办公。 新地址为：上海张江祖冲之路887弄88号502室。 ==========================================================
-
Maarten Lankhorst -
shenyue